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ABSTRACT 

To provide information to the Congress in its 
continuing debates over improving science and mathematics (S/m) 
education. as well as to education .policymakers, the General 
Accounting Office (GAO) synthesized past evaluation and research 
studies and- used other methodologies to examine four issues: (1) the 
nature of the problem and its remedies; (2) prospects for upgrading 
existing S/M teachers; (3) prospects for reducing the S/M teacher 
shortage by retraining teachers from other, subject areas; and (4) the 
current ability to evaluate the quantity and quality of S/M teaching. 
Observations on the implications of this Work for S/M education 
legislation are included. GAO does not find evidence that training, 
programs to upgrade existing S/M teachers will improve teaching ' * 
effectiveness. Such programs are a prominent 0art of proposed federal 
legislation. Teacher shortage problems, which are also addressed in 
the proposed legislation, may be easier to solve. Programs to retrain 
teachers of other subjects to teach science and mathematics classes 
seem to be one viable solution to technical teacher shortages. In 
addition, gaps itt information available to policymakers are so severe 
that GAO could no,t determine if there are net nationwide shortages of 
S/M teachers and if the quality of technical teaching has declined in 
recent years. (JN,) 
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GAO does not find evidence that training pro- 
grams to upgrade existing science and mathe- 
matics teachers wttt improve teaching effective- 
ness, "Such programs are a prominent part of 
proposed federal, legislation. , ' 

Teacher shortage problems, which are also ad- 
dressed in the proposed legislation, r/iay be eas- 
ier to sotve. Programs to retrain teachers of ether 
subjects to teach mathematics and science 
classes seem to be one viable solution to techni- 
cal teacher shortages. 

Gaps in the information available^ policymak- 
ers are so severe that GAO could not determine 
(1) if there are. net nationwide shortages of 
science and mathematics teachers and (2) i f the 
quality of technical teaching has declined in 
recent years, 
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GENERAL ACCOUNTING OFFICE 
REPORT 



NEW. DIRECTIONS FOR FEDERAL 
PROGRAMS TO AID MATHEMATICS 
&ND SCIENCE TEACHING 



DIGEST 

The status of mathematics and science 
education in the public schools became a iwjor 
Issue by i?83* GAO synthesized past evaJua- 
tion and research studies and used other meth- 
odologies to examine {l) the nature cl the 
problem and its iTemedles, (2) the prospects 
for upgrading existing mathematics and science 
teachers, (3) the viability o£ retraining 
teachers of other subjects to teach science 
and mathematics, and (4) priorities fbr evalu- 
ation in mathematics- and science teaching* 

W HAT IS THE NATURE OF THE 
MATHEMATICS AND SCIENCE TEACHING 
PROBLEM? 

If the sweeping reexamination of mathematics 
and science education in the 1350 l s was funda- 
mentally driven by national security concerns, 
national attention in the 1930 's seems powered 
by international economic competition* Con- 
tinued high levels of unemployment and visible 
losses of -automobile and consumer electronics 
markets, for example, have fueled proposals 
for action* 

Some observers link national economic prosper- 
ity to improved 5;cience and mathematics educa- 
tion aimed at achieving growth through tech- 
nology, but others have opposing views of 
.the goals of education and place different em- 
phases on the educational needs of the future 
United States work force* The apparent 
national consensus on the need for educational 
reform thus obscures significant disagreement 
with regard to the dimensions and direction of 
that reform* Critics differ in their emphasis 
on mathematics and science as opposed to other 
subjects and on the education of the elite {or 
most able) as opposed to the education of all 
students* Even within the fields of mathemat- 
ics and science education alone, it is diffi- 
cult to pursue goals of improving education 
for the'most able and the average student 
simultaneously* To do so would be to increase 
the need for more teachers, which may require 
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relaxing standards, and to increase the need 
for better teachers* which may requinre raising 
standards * (pp * 7-10 ) 

Those vho place priority upon technical 
education are concerned ov«r evidence of 
problems with the qurntity and quality of 
mathematics and physical science teachers in 
the United States public schools: 

— 42 out of 45 responding states reported a 
mathematics teacher shortage in 1982; 

—surveys show a drop of 64 percent and 33 
percent in Mathematics education and science 
education eraduates* respectively* from 1971 
to 1981* aLthough reduced enrollments had cut 
the, production of new teachers 39 percent 
across all fields; * 

"about half of recent bachelor degree gradu- 
ates who are teaching science and mathematics 
are not certified or eligible for certifica- 
tion in the field they are currently 
teaching* {pp* 10-14 ) 

Legislation to upgrade science and mathematics 
education— House of Representatives 1310 and 
Senate 1285 — is being considered by the 
Congress* (pp* 14-18) 

WHAT ARE THE PROSPECTS FOR IMPROVING 
THE EFFECTIVENESS OF TEACHING THROUGH 
TRAINING TO UPGRADE EXISTING MATHEMATICS 
AND SCIENCE TEACHERS? 

GAO does not find evidence that training pro- 
grams to upgrade existing rrfathematics and 
science teachers will produce results in terms 
of improved student achievement* 

GAO first examined the prospects of such train- 
ing programs by reviewing the experience of 
teacher institutes funded V>>\ the . National 
Science Foundation <NSF) from the mid-1950 l s to 
the early 1970's, A search for studies of the 
impact of teacher attendance at thsse insti- 
tutes upon the subsequent academic achievement 
of students of the participating teachers pro- 
duced only one study that met GAO's minimum 
criteria, and it showed mixed results* {pp* 
22-25 and 31-32) 

o 
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GAO then searched for related evidence* 
focusing on the fact that subject matter train- 
ing was a major element of the NSF institutes* 
The general research in the 1970 's failed to 
show any consistent relationship between the 
extent of teachers' knowledge and subsequent 
student learning. (pp. 32-34) 

More recent "process-product" research sug- 
gests that student performance can be improved 
by training teachers to manage instructional 
programs and student behavior. The results of 
this emerging research area are promising, but 
process-product research has not yet focused on 
the secondary school level. (pp. 34-36) 

HOW VIABLE A SOLUTION TO SHOR T AGES OF 
MATHEMATICS AND SCIENCE TEACHERS IS 
RETRAINING TEACHERS CERTIFIED IN 
OTHER FIELDS7 

GAO finds that programs to retrain teachers 
from other subjects to teach science and math-* 
ematics classes seems to be one viable solution 
to the technical teacher shortage. " Early re-* 
suits from 11 programs show that teachers apply 
for admiss ion i enroll i and are starting to com** 
^gplete retraining programs. There is little 
Reason to doubt that most program graduates 
will become certified mathematics or science 
teachers. (pp. 51-55 ) 

It is too early to determine the quality of 
retrained teachers, but obtaining certification 
indicates a threshold of quality. The results 
reported in the previous section suggest that 
further upgrading of teacher quality may not 
lead to improved student achievement. 

GAO finds that retraining programs sponsored by 
state education agencies (SEA*s) and local edu- 
cation agencies (LEA* s) tend to have higher re- 
tention rates than university programs. SEA 
? and LEA programs provide funding for selected 

^ teachers to attend college classes, while re- 

training programs fully controlled by univer- 
sities charge full tuition and do not systema- 
tically provide student financial assistance. 
The greater success of SEA and LEA programs 
seems to be due to a combination of the funding 
provided and their more stringent selection 
process. (pp. 46-52 ) 
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The Houston school district appears to have 
been remarkably successful in alleviating the 
shortage of certified technical teachers. 
Retraining programs combined with other incen- 
tives and bonuses contributed to a sharp re- 
duction in the secondary science and mathe- 
matics teacher shortage between 197'S and 1982. 

GAO finds wide variation in the length of time 
to complete retraining progarams ranging from 
9 months to 3 years. Shorter programs insist 
on more mathematics or science courses as ad- 
mission requirements and provide for full-time 
attendance during at least part cf the 
program. {pp* 50-53) 

WHAT WILL BE NEEDED TO 
IMPROVE EVALUATIONS OF THE 
QUALITY AND QUANTITY OF MATHEMATICS 
AND SCIENCE TEACHING? 

Data do net exist to determine with confidence 
whether or not there is a net nationwide short^ 
age of mathematics and physical science teach- 
ers. Information on the quality of mathematics 
and science teachers — and whether or not. the 
quality of technical teaching has declined — is 
also flawed. , (pp* 10-14) 

In addition to improved data on the*e critical 
issues, GAC poses evaluation questions for 
consideration in the areas o* both quantity 
and quality of mathematics and science 
teaching. ( pp. 57-63 ) ■ 

OBSERVATIONS 

1. GAO's analysis raises questions about 
approaches to upgrade the quality of mathemat- 
ics and science teaching that have substantial 
teacher training components. This report sug- 
gests that programs geared at upgrading exist- 
ing mathematics and science teachers may not 
produce results in terms of improved student 
achievement. Upgrading training may be best 
focused upon uncertified teachers now in or 
coming into the mathematics and science class- 
rooms. Past research may rot be germane to * 
this group. This approach would differ from 
the prior NSF strategy and would require 
careful planning for successful NSF implemen- 
tation. It may be more productive to concen- 
trate resources upon filling mathematics and 
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science teacher vacanci, s. GA3 found evidence 
that retraining programs are one viable solu- 
tion to the' technical teacher shortage. Thtt-- 
quantity or shortage problem may be more suc- 
cessfully dealt with because it is a simpler 
problem. {pp. 65-66 ) 

2. Programs to- retrain teachers for mathemat- 
ics and science classrooms are highly variable 
in length but can be controlled by policy in- 
tervention. Shorter, full-time retraining pro- 
grams can produce certified teachers sooner, 
but full-time programs will be more costly, 
since tfiey involve both scholarship and sub- 
sistence costs for the teachers being retrained 
plus salary expenses for replacement teachers 
during , the retraining period. In the absence 
of scholarship and subsistence "payments, full* 
time programs seem to attract few students* 

(p. 66) 

3. The strategy in v the proposed mathematics 
and science legislation of requiring linkage 
between universities and school districts in 
training and retraining programs may be produc- 
tive since the SEA and LEA retraining programs 
tend to experience higher retention rates than 
university programs. (pp* 66-67) 

4. The efforts of process-product' researchers 
to identify effective teaching, behaviors- and to 
develop teacher training programs around^thoee 
findings offer promising possibilities for con- 
sideration by the Department of Education and * 
other funding sources. The approach has been 
limited to mathematics and reading and could be 
attempted in science teaching. Since the bulk 
of the effort has been conducted at the elemen- 
tary grade levels, it may be' useful to devise a 
secondary level research program. (p. 67) 

5. Data are not available to determine whether 
or not there is a* net shortage of mathematics 
and science teachers or to assess the quality 
of teaching in those* fields* Thex^ are two 
corresponding information needs which result* 
First, with respect to ,the size of any short- 
age, is the neepl for adequatj d^ta at both the 
national and state levels on the extent of the 
shortfall by subject each year* Second, with 
respect to quality, is the need for an adequate 
assessment of the knowledge levels of mathemat- 
ics and science teachers and whether or not 



they are out-of-date in their subject areas* 
These information gaps hinder both the enact- 
ment and administration of. effective federal 
remedies and retard appropriate evaluation of 
the success of new federal remedies* 

Limited evaluation support provided to the 
programs in the proposed federal mathematics 
and science education aid- bills, combined with 
budgetary pressure limiting the availability 
of other agency funds, suggest that it may 
never be known whether or not those programs 
are effective. (pp. 67-68) 

AGENCIES* COMMENTS 
AND GAP'S RESPONSE 

NSF, the Department of Education, the office of 
Science and Technology Policy, and the office 
of Management and Budget commented on a draft 
of this report* The agencies generally charac- 
terized the report as useful and addressed to 
issues of major importance* 

The agencies raised a concern about GAO**s find- 
ing that programs to upgrade science and mathe- 
matics teachers are not likely to produce 
results in terms of improved-student achieve* — 
ment. The few facts cited against this find- 
ing, however, gave GAO no reason 'to alter the 
report. The agencies generally agree, with 
GAO's observations on the need to remedy the 
lack of evaluative information for shaping 
effective federal programs to improve mathemat- 
ics , and science education* (app* III) 
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CHAPTER 1 



INTRODUCTION 

The status of mathematics and science education in the 
public schools became a major issue in 1983. Concerns were 
raided over the level of student achievement in these subjects 
and ,over the shortages of teachers certified to teach mathematics 
and science in secondary schools. The dialogue over mathematics 
and science education expanded into a wider review of the quality 
of teaching and the quality of our schools. Numerous commissions 
issued reports recommending a variety of educational reforms. 
The Congress considered legislative proposals for federal * 
assistance for mathematics and science education. 

In this report, we address four science and mathematics edu- 
cation issues: 



— the nature of the problem and its remedies; 



— the prospects for upgrading existing mathematics and 
science teachers; 

— the prospects, for reducing the mathematics and science* 
teacher shortage by retraining teachers from other subject 
areas ? and 

— the current ability to evaluate the quantity and Quality 
of mathematics and science teaching. 

We then make observations on the implications of our work 
for mathematics and science education legislation. Our report 
utilizes the evaluation synthesis an<3 other methodologies. Its 
purpose is to provide information to -the Congress in continuing 
debates over improving mathematics and s.cience education, ,to t edu*- 
cation policymakers at all levels of government, as well as to 
the education profession. 

BACKGROUND 

Some three "decades ago, this country became alarmed over the 
status of mathematics and science education and our ability to 
keep pace technologically with the Soviet Union. The release of 
a 1955 report, Soviet Professional Manpower by Nicholas DeWitt, 
detailing Russian scientific advances, and the subsequent launch- 
ing of Sputnik I by thft Soviet Union in 1957 stirred demands for 
more education in the sciences. 1 While £Xtensive documentation 
was not available* there was widespread agreement that there were 
scientific and technical manpower shortages. How could those 
shortages be eliminated? There was evidence that about half the 
most able high school students were not entering college. It was 
argued that many students did not enter college in technical 
fields because of problems of poor teacher preparation and obso- 
lete curricula. 2 One answer to poor teacher preparation by the 
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then recently created National Science Foundation (NSF) was a 
program of summer instruction for high school teachers, ^ 

'> 

Another major federal initiative, the National Defense Edu- 
cation Act of 1958, authorized a variety of activities including 
the purchase of laboratory equipment, upgrading guidance and 
counseling services, improvement of foreign language teaching, 
and improvement of education statistics, among others, 4 j n the 
1960 's, fedet .*! education aid ..moved toward a concern with equal- 
ity of educational opportunities for the poor and for racial and 
language minorities, In the 1970 's, in the wake of university 
student protests about curriculum content, among other things, 
educators became concerned over, the relevance of education, 
Schools experimented with ways of increasing student choice of 
courses, reducing competition among students in an attempt to 
individualize educational experiences, and increasing the influ- 
ence of students and parents in school decisionmaking* 

By 1983, the status of mathematics and science education in 
the public schools again emerged as an issue, Among .the most 
frequent concerns raised are 

— declining student achievement test scores in mathematics 
and science, The widely used Scholastic Aptitude Test, 
for example, shows a steady and appreciable decline in 
the mathematics subscore from 1962-64 to 1980* The Na- 
tional Assessment of Educational Progress found a steady * 
decline in the science achievement of 17-year-olds since 
the initial testing in 1969, although some other tests 
show student science achievement to be more stable. 
There is some evidence that the' declines may bave 
bottomed out, but they have not yet reversed, 6 

* 

— reported shortages of teachers in mathematics and some 
science fields, The Department of Education found short- 
ages of technical t^achers, such as a shortage of 900 
mathematics teachers and 600 physical science teachers as 
early as 1979,' A survey of college placement officials 
published in 1983 found a considerable shortage of teach- 
ers in mathematics, physics, and chemistry,** 

— the apparent decline of American technology from its 
preeminence in the 1960 's* Many technology-based 
industries are now in second or third place vis-a-vis 
foreign rivals; this has focused attention on our 
mathematics, science, and engineering capabilities, 9 , 

A succession of reports by commissions and others made 
recommendations on mathematics and science education either cen- 
trally, as in the case of the National Science Board Commission 
on Precollege Education in Mathematics, Science, and Technology, 
or as part of a broader review of the> quality of teaching and 
schooling, Some of the more prominent examples of the latter are 



the reports of the National Commission on Excellence in Education 
and the Task Force on Education for Economic Growth* 10 



On March 2, 1983, ttfe House passed House of Representatives 
1310, the Emergency Mathematics and Science Education and Jobs Ac 
The Senate is also considering similar legislation, senate 1285* 
Both bills authorize a variety of programs for improving mathe- 
matics and science education. Which we will review in chapter 2* 

OBJECTIVES, SCOPE, AND METHODOLOGY . 

The four study questions we deyelop in this report are 
listed in table 1# The first question — what is the nature of Che 

Table l 

Mathematics and Science Education Questions and Subquestions 



QUESTION 


SUB0UEST10N "1 


1-0 What is the nature of the math- 
ematics and science teaching 
problem? 


1-1 What will be the effect of new 
technologies on the need for 
science and mathematics 
education? 

1-2 What are the problems in the 
quantity and quality of mathe- 
matics and science teachers? 

1-3 What remedies have beefi proposed 
* for these problems? T 


2-0 What are the prospects for im- 
proving the_qualijty_a.f teach inq 


2-1 What was the extent and nature 
of participation in the NSF 


through training to upgrade 
existing mathematics and 
science teachers? 


institute program? 

2-2 How effective were the NSF 
institutes? 

2-3 Are more "knowledgeable teachers 
more effective? 

2-4 What does recent classroom 

research tell us about how to 
improve student achievement? 


3-0 How viable a solution to 

shortages .of mathematics and 
science teachers is retraining 
teachers certified in other 
fields? 


3-1 What are tho characteristics of 
the new* retraining programs? 

3-2 " How effective are the new 
retraining programs? 


4-0 What will be needed to improve 
evaluation of the quality and 
quantity of mathematics and 
science teaching? 


NONE 



mathematics and science teaching problem? — is largely conceptual* 
Rather than repeat the exhaustive recitat ion of the problem that 
is available elsewhere, we introduce the problem and frame it in 
the context of concerns about United States economic strength *** 
We note the quantity and quality components of the teaching prob- 
lem and note other school improvement approaches, We limit this 
report to the public schools because of the difficulty in obtain- 
ing data on private schools, 

The second question — what are the prospects for improving 
the quality of teaching through training to upgrade existing 
mathematics and science teachers? — involves a synthesis of evalu- 
ations from NSF teacher institute programs that spanned two dec- 
ades. We chose to examine this issue because past NSF experience 
with training to upgrade mathematics and science teachers could 
have an important practical application since suct^upgrading is a 
prominent feature of the proposed legislation, We discuss our 
methodology of searching for and selecting evaluation studies in 
chapter 3, Studies on the extent and nature of participants 
drawn to such programs and the available evidence on effective- 
ness worthy, of use in congressional decisionmaking are reviewed* 
We then draw on a body of relate'd research dealing with the links 
between teachers' knowledge and the achievement gains of their 
students* More recent approaches emphasizing effective teaching 
through systematic classroom management are then assessed* The 
major limitation of this approach is the weakness of the evalua- 
tion studies of the NSF institutes* 

The third question seeks to determine the viability of re- 
training teachers certified in other subject areas as a remedy 
to the shortage of mathematics and science teachers* We chose * 
examine this is$ue because it was the emphasis of the administra-* 
tion bill (which was introduced but not yet acted upon by either 
the House, or the Senate) and becaus e some retraining programs are 
being operated through nonfedera'l furiS'iKg" sources* Our approach 
began with a literature search but t the recency of these programs 
signifies a paucity of available data and literature* We turned 
instead to Identifying a sample of such programs, using a method^ 
ology.we describe in chapter 4, We then explored^these programs 
through telephone interviews supplemented, by available written 
materials provided by-direQtors of the programs, Since these 
programs are too new to -nake effectiveness or impact evaluations 
possible, the focus of our *data collection was upon the program 
goals, length of the programs, and evidence, of, feasibility or 
plausibility of the programs to date as one solution to the prob- 
lem of the quantity of mathematics „and science teachers* 

The fourth qnestipn — how can the quality and quantity of 
mathematics and ^science teaching be evaluated effectively?— con- 
cerns an issue that repeatedly troubled us during the conduct of 
this review, The availability of information 4 on the extent of . 
needs in .the quality and quantity of mathematics and science 
teaching is a fundamental prerequisite to sound public policy 



planning* Our method here draws upon our experience in 
addressing the other study questions _ and upon the developments in 
the field of program evaluation* 

We discuss the questions in the order in which they appear 
in table 1 and in the preceding discussion. Finally, in the last 
chapter we draw together the findings of this review to form 
observations for the implementation of new federal mathematics 
and science education legislation* 

NOTES 

^Nicholas DeWitt, Soviet Professional Manpower (Washington, D.C. z 
National Science Foundation, 1955) * See also Milton Lomask, A 
Minor Miracle; An Informal History of the National Science 
Foundation (Washington, D.C. i National Science Foundation, 
1976), p. 122. 
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CHAPTER 2 



WHAT IS THE NATURE OF 

THE MATHEMATICS AND SCIENCE 

TEACHING PROBLEM 7 

In this chapter, we introduce the context of the concern 
over mathematics and science education* especially the issue of 
what technical education needs the United States may have in 
order to secure our future in international economic competition, 
This is not* of course* the only goal of technical education; 
however* the role of technical education in advancing economic 
growth is the connection with this chapter, 

We find that the apparent national consensus on the need 
for educational reform obscures significant differences in per- 
ceptions of educational priorities necessary to achieve greater 
economic growth* We "find that many problems raised about the 
state of mathematics and science education can be classified as' 
pertaining to either the quantity or quality of teaching, We 
review the evidence on problems in both the shortage (or quan- 
tity) and the quality of mathematics and science teachers, We 
summarize the remedies to those problems proposed in federal leg- 
islation as well as by state and local educational agencies, 

WHAT WILL BE THE EFFECT OF NEW 
TECHNOLOGIES ON THE NEED FOR 
SCIENCE AND MATHEMATICS EDUCATION 7 

If the sweeping reexamination of mathematics and science * 
education in the 1950 's was fundamentally driven by national . 
defense concerns* national Action in the 1980*s seem# powered by 
international economic competition, Continued high levels of un- 
employment and visible losses of automobile and consumer elec- 
tronics markets to foreign competition have fueled proposals for 
action* which include^upgrading mathematics and science educa- 
tion* Since the nature of the mathematics and science education 
remedies is so closely linked to the nature of the perceived prob- 
lem* in this section we examine different directions or models of 
future economic change and tneir educational implications, This 
analysis shows that there aro great differences in views about 
what the future mix of jobs will be* 

High tech model 

The high tech model suggests that growth in high technology 
industries is one solution to our economic problems, President 
Reagan said in his state of the union address on January 25* 
1983* 

"as surely as America's pioneer spirit made us the Indus-* 
trial giant of 20th century* the same pioneer spirit today 



is opening up another vast frontier of opportunity — the 
frontier of high technology* In conquering the frontier we 
cannot write off our traditional industries* but we must de- 
velop the skills and industries that will make us a pioneer 
of tomorrow* 

■ * 

The Task Force on Education fot Economic Growth predicts 
that ".the conditions that concern us today — swiftly advancing 
technology; economic competition in a global arena; the sudden 
obsolescence of skills — will be even more intense tomorrow* "2 
yhe advance of technology in the workplace will extend, the Task 
Force ar9 id s, from word processors in offices to sophisticated 
weapons systems in the armed forces to replacing lift operators 
with computerized conveyor system operators* 3 * * 

In congressional debate over the mathematics and science ed-' 
ucation legislation, Congressman Ford o # £ Michigan observed that 

"Management analyst Peter Di'ucker predicts that through the 
next two decades 10 t;o 15 million manufacturing jobs will 
* disappear in America* * * * [I]t is clear th^t the new jobs 
will be in areas such as computer technology, robotics, 
fiber optics, genetic engineering and health care** - ^ 

Congressman Ford continued, noting shortages of trained and 
skilled workers in the labor force: 

"Shortages already exist or are anticipated for engineers, 
nurses , computer service technicians, and machinists, among 
others* For example, the American Electronics Institute 
estimates that industry will need nearly 200,000 new en- 
gineers by 1985; while universities, given their current 
faculty, can supply only 70,p00* * * * The Defense Depart- 
ment estimates that its contractors will need 71,000 more 

£<Dii^t^ir~^^a^ el^ctricat-engineers-and 

110,000 more machinists, tool and die makers and metal mold- 
ers by 1987* " 5 

Educationally , v this model suggests more training in mathe- 
matics, computer science, and technical applications throughout 
the labor force* 6 Thus, schools must reverse the decline in stu- 
dent achievement' in mathematics and science as well as upgrade 
the mathematics and science curricula and teacher preparation* 
The concern is not limited to the elite future leaders in science 
and technology but extends to a concern over upgrading the scien- 
tific literacy of the population more generally* Lagging student 
achievement in science and mathematics has led ; to the fear that 
"We are raising a new generation , of Americans that is scientific- 
ally and technologically illiterate*"^ The high te.ch model is 
the intellectual underpinning of the proposed federal legislation 
to provide aid for mathematics and science education* 
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Minimal high tech model 



Others argue that the impact of high technology on Ameri- % 
ca's economic future has been overstated. While the percentage 
growth in some high tech occupations may be dramatic* the fact 
that these occupations are n<$w relatively rare means that even a 
large percentage increase will tr&nslate into a modest number of 
new jobs. For example, while jobs for computer systems analysts 
may increase by over 100 percent from 1978 to 1990* only 2d0*000 
new jobs will result. In contrast* over 600*000 new janitorial 
jobs will be created. There will be more new janitorial jobs 
than the combined total of new jobs in the five occupations with 
the highest percentage growth rates. These observers also argue 
that the impact of high tech in transforming existing jobs is 
being exaggerated or even distorted. As examples, they point to 
word processors as reducing needed skills for office work and to 
technological advances in printing as reducing the skill levels 
of those who remain in newspaper composing rooms. ® 

Educationally* we would expect under this model somewhat 
more limited concern over mathematics and science education 
advances. Mathematics and science education concerns present 
would logically be centered about the elite — an admittedly * • 
sizable elite, but an elite nonetheless — that will be Reeded to 
secure the high tech side of our future- Other educational 
implications would vary depending upon the observer's vision of 
the economic future- 

Learning to learn model 

Finally* it has been argued that the crucial growth of 
productivity depends on the utilization of knowledge from all . 
sources to meet needs. Technological progress may make new 
achievements possible but a nation* s economic strength is more 
depend entrnm-the^bi^ity^o^^ "reach out and 

grasp tomorrow's technologies and markets before competitors.' 1 ^* 
At the same time, firms must strive to "improve today's products 
and production processes. " if The latter consists of numerous 
modest changes "soma from short-term R&D but most from experience 
with the technology . \ . to better satisfy market require- 
ments."^ The increasing need for effective technological inno- 
vation suggests a value of interdisciplinary collaboration under 
conditions that Encourage the flow of new ideas and the ability 
to confront and work through differences. 12 

* * 



How do you educate for grasping tomorrow's technologies 
while improving today* s productsand production processes? The 
Task Force on Education for Economic Growth links education needs 
to four types of jobs. First* unskilled jobs such as hauling and 
janitorial work can be performed by people with less than today's 
basic skills. Second* basic jobs such as clerks in noncomputer- 
ized stores require today's basic education. Third* "learning- 
to-learn 1 ' jobs including most factory and service industry jobs 
will require teaching people how to acquire new skills of 
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analysis and problemsolving. Fourth, professional jobs 
require ,"learning-to-learn" skills and more sophisticated 
intellectual skills as well. Real chances for upward mobility 
"will increasingly be reserved for those with ' learning-to-learn ' 
skills; not just the ability to read, write and compute at a 
minimal level, but more complex skills of problem solving, 
reasoning, conceptualizing and analyzing. "13 it is such general 
skills, rather tharv specialized training, that are proposed 
for a future in which work requirements are expected to change 
abruptly, and it will become increasingly difficult to predict 
beyond general trends what specific jobs will be in demand. ^ 

Educationally, all this suggests the need to "raise both 
the floor and the ceiling of achievement in America, improving 
educational attainment for the most able students and for other 
students as well." 15 The Task Force on Education for Economic 
Growth calls for upgrading basic skills. Competency in reading 
may well extend beyond literal interpretation to include the 
ability to analyze and summarize as well as to interpret passages 
inferentially . Mathematical compatency may well come to include 
more complicated ^computing and problemsolving skills. Writing 
competency may capture the ability to gather and organize 
information coherently . !6 

The National Commission on Excellence in Education reached 
a similar conclusion, arguing tbat recent efforts to improve 
mathematics and science education are 

"but a start on what we believe is a larger and more educa- 
tion^ My encompassing need to improve teaching and learning 
in f i elds strtfh as English, history, geography, economics, 
and ioreign languages- We believe this movement must be 
broadened and directed toward reform and excellence through- 
out education. "1^ 

It may be observed from the above discussion of recent re- 
ports that the apparent national consensus on the need for educa- 
tional reform obscures significant differences in perspectives on 
the future educational needs of this nation. These perspectives 
place varying emphasis on technological as opposed to other sub- 
jects and on education of an elite as opposed to that of all 
students . Even v/ithin the fields of mathematics and Science edu- 
cation* alone, it is difficult to pursue goals of improving educa- 
tion for the elite and the average student simultaneously. To do 
so would be to increase the need for more teachers, which may re- 
quire relaxing standards, and to increase the need for better 
teachers, which may require raising standards . 

WHAT ARE THE PROBLEMS IN THE QUANTITY AND 
QUALITY OF MATHEMATICS AND SCIENCE TEACHERS? 

Many of the problems raised about the state of mathematics 
and science education in the elementary and secondary schools 
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can be classified as pertaining to the quantity or quality of 
teaching- 

Quantity * * 

One major concern, was the evidence of shortages of 
secondary school teachers in mathematics and some science sub- 
jects* An annual survey of college placement officials, found 
that the officials believed that there were teacher shortages in 
mathematics, physics, chemistry, and earth science in 1978 and 
greater shortages in all those fields by 1983. 18 An annual sur- 
vey of state science supervisors between 1980 and 1982 found that 
they believe/the shortage of science teachers to increase or stay 
about the same each year* By 1982, 42 out of 45 responding 
states reported a mathematics teacher shortage and 42 out of 47 
responding states noted a physics teacher shortage. Science 
teacher shortages seem to be modest or nonexistent in biology and 
general science. 19 Another 1983 survey showed a science and/or 
mathematics teacher shortage in 38 states. 20 

United States^ Department of Education surveys show a drop of 
64 percent and 33 percent in mathematics education and science 
education graduates, respectively, at the bachelor degree level 
in 1981, compared with a decade earlier- The average for all 
fields of education was a drop of 39 percent. 21 The general de- 
crease reflects the impact of declining student enrollments in 
many states created by a drop in birth rates. ' 

Much of the shortage has been met by assigning teachers 
with other specializations to mathematics and science classrooms 
on an, emergency or provisional basis. A survey of 1979-80 bache- 
lor degree recipients who were teaching in May 1981 found that 56 
percent of those teaching science and mathematics were not 
certified or eligible for certification in the field in which 
they were currently teaching. This compares with 22, percent for 
all teachers and 26 percent for all specialty teachers. 22 ^ 
sample of 1,000 secondary school administrators surveyed by the ' 
National Science* Teachers Association in December 1981 found that 
half the newly employed science and mathematics teachers were re- 
ported by administrators to be "unqualified 1 * to teach science and 
mathematics .23 * 

Although it would appear obvious that recent actions by 
states to raise graduation requirements in mathematics and sci- 
ence will, increase enrollments and consequently create additional 
shortages of mathematics and science teachers, i n reality such 
shortages may not occur* States are increasing the science and 
mathematici requirements. 0ne"1983 analysis found that 11 states 
have increased graduation requirements in mathematics since 1980, 
while 11 others were seriously considering increased require- 
ments. Three states had increased science requirements while 7 
had increases under serious consideration- 24 Th e recent commis- 
sion reports recommending stiffer graduation requirements are 
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likely tq Accelerate this trend. ' Yet* high school enrollments 
will still be declining as a result of lowered birth rates in the 
1970 's. Between 1935 and 1990, the diop will be from 13.6 
million to 12.4 million, a decline of 8*8 percent or over 1.7 
percent per year. 25 These declines will at least partially 
cbmpensate for increases in demand for teachers due to increased 
graduation requirements. Even the increases due to stiffer 
graduation requirements may be illusory* One state official told 
us that their local school superintendents were informally polled 
about recently increased state requirements and none anticipated 
any impact because existing local requirements exceeded the new 
state minimum requirements in all cases. Future teacher short- 
ages or surpluses in technical subjects are farther influenced by 
the following factors* * (1) number of persons newly certified to 
teach in mathematics and each science field each year; (2) turn- 
over of teachers due to retirement* new employment outside of the 
classroom* death, &nd other reasons; (3) return of former teach- 
ers to the labor force? and (4) "market* solutions to, shortages 
such as increaned undergraduate enrollments* in mathematics and 
science education programs as a result of the publicity over 
shortages of teachers in these ^fields- A study that systematic- 
ally examines all or most of these components or factors in order 
to assess the current and future supply and demand for mathemat^ 
ics and science teachers has — to our knowledge and at the date of 
this writing— simply not been done. 

The studies we have cited above have made various compro- 
mises in their research, designs in order to attempt to measure 
the supply and Remand in mathematics and science teaching. We 
found the resulting study designs and the data available from 
these studies to be seriously flawed. For example, both the 
survey of teacher placement officers and the survey of state 
science supervisors are based on simple opinion checklists. The 
respondents are asked to indicate for each field (38 fields for 
J bhe form e r and 6 tec hnical subjects for the latter) the shortage 
'or surplus of tfeaHhers on ~ttv€^o^fit~a'ttl:&^^* ~ - 

There are sources of possible bias in data from such 
studies* At one ^evel* the increased awareness of a shortage 
problem may increase the amount of reporting of shortage prob- * 
Jems when no actual increase of the problem is occurring* At 
another level, the respondents providing the basic data and per- 
haps the groups sponsoring the studies cannot be said to be with- 
out self-interest* Findings suggesting that shortages are 
greater rather than ,smaller t would appear to*enhance the role of 
science supervisors and teacher placement officials* One 
national expert in the quantity of .mathematics and science 
teachers told us that some state officials have admitted 
of f- the- record that they have shortages only in some locations 
within the state rather than statewide shortages* 

We conclude* therefore* that the data do not now exist to 
determine With confidence whether or not there is a net nation- 
wide shortage of mathematics and physical science teachers. 
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Surely there are some local shortages, We did not search for 
state surveys so there may be well-documented shortages in indi- 
vidual states, Nationwide data showing that a high proportion of 
new mathematics and science teachers were not Certified in their 
current teaching field certainly suggest a shortage situation* 
Yet other key questions — how poorly prepared are the3c teachers 
and do they subsequently obtain certification? — remain unan- 
swered, Are we concluding that there is no teacher shortage in 
science and mathematics? Certainly not, But we do conclude that 
there exists no reliable statistic on the current or future * < 
national shortage of teachers in mathematics* physics* or other 
science fields, 

Quality 

i 

Most of the concerns about the quality of mathematics and 
science teaching are inferential, based on surrogate measures of 
quality such as certification or on characteristics of teachers 
in all fields rather than technical teachers alone, The promi- 
nence of uncertified teachers among the new recruits to mathemat- 
ics an<i science teaching as cited above raises questions about* 
the quality of their preparation and their classroom effective- 
ness. Further* education generally has attracted for undergrad- 
uate majors those with low scores on standardized achievement 
tests, College bound seniors intending to st.udy education ranked 

6 out of 29 majors on the 1981 Scholastic Aptitude Test on both 
the mathematics and verbal subtests, Only home economics (27)* 
ethnic studies (28)* and trade and vocational (29) ranked lower, 
Prom 1973 to 1981, verbal scotes for prospective education majors 
dropped 27 points while mathematics scores fell 31 points, Both 
declines exceeded the national average for all fields of 21 and 
15 points* respectively, Scores of education majors on the Grad- 
uate Rei ord Exam were substantially lower than scores of majors 
in 8 other professional fields in 1975-76, 2 ? These findings may 
or may not signal a further decline from the low achievement 
levels of_education majors found in similar analyses during the 
19 50 1 s , One " s tudy of 10700 0 col l eg e™graduaire s~ r eported— i n- 1 9 5 4- — 
that education ranked 17 among 20 fields of study, 29 

A followup study of the National Longitudinal Study of 
1972 High School Seniors presents a disturbing portrayal of 
those who remain in teaching, Verbal and mathematics subtest 
scores in the Scholastic Aptitude Test were examined for those 
who 7 years later had graduated from college, The results were 
essentially identical for both subtests, The highest scoring , 
group — averaging 496 on the verbal subtest — was the nonrecruits* 
those who did not major in education and never taught, Lower 
scoring and- essentially tied were the "defectors 11 (averaging 462) 
and the "confirmed defectors** (averaging 460), The former were 
teachers who do not intend to teach at age 30* The latter were 
ex-teachers* At the bottom — averaging 432 — were'the "committed 
teachers*" those who taught and intended to be teaching at age 
30, The rankings are^fche same on the mathematical reasoning 



13 28 



subtest. Nonrecruits average^ 537* defectors and confirmed 
defectors 483* and committed teachers 470. 2 ^ In short, not only 
are the academically less able attracted to teaching school but 
the least able are retained in the field. 

One temptation is to raise standards for admission to 
undergraduate education programs at colleges and universities. 
States have been moving rapidly to require teacher testing for 
admission to teacher education programs or at some point in the 
certification process. 30 Yet in the National Longitudinal Study 
samples , excluding those in the bottom 20 percent in measured 
verbal ability (combining nonrecruits and recruits to teaching) 
would remove 30 percent of the teachers and 34 percent of the 
"committed teachers-** In mathematical reasoning ability* the 
percentage's are 30 and 29, respectively . 31 Excluding this bottom 
fifth of teachers from the field would raise the quality 
level — at least as measured by a standardized test — but Would 
reduce the quantity of teachers available and thus create new 
shortages. Efforts could be undertaken to recruit students in 
the upper quarter of their high school class as mathematics and 
science teachers* but persons with this level of ability may 
simply not be interested in teaching careers. 32 

These findings raise the question of the relationship be- 
tween a teacher's academic ability and teaching effectiveness. 
Do i*re need brighter teachers or are they inherently impatient, 
lacking in empathy, or likely to be bored by working with chil- 
dren?" ,We explore the relationship between teacher knowledge 
and teaching effectiveness as well as describe recent research on 
effective teacher classroom behavior in the next chapter. 

WHAT REMEDIES HAVE BEEN ; 
PROPOSED FOR THESE PROBLEMS? 

Observers frequently cite two major causes of the difficult 
ty in attracting more 'and better people to teaching mathematics 
and science- One is the low level of teacher salaries. One 
study found that the 1981-82 average starting salary for bache- 
lor degree teachers was $12,769.34 a study of approximately 
200 companies found that industry was offering those with a 
bachelor degree in mathematics or statistics an average start- 
ing salary of $18,600, or about $5,800 'more than starting teach- 
ers. Salaries in chemistry ($19,536) and computer sciences 
($20,364) were even higher. 35 it is not surprising that a survey 
of college graduates newly qualified to' teach in May 1981 found 
that 27 percent of mathematics education graduates—nearly twice 
the average for all education fields — did not even Apply for a 
teaching job. 36 

There are some efforts to remedy this salary disadvantage 
through bonuses for new teachers in mathematics and science and 
other shortage areas. Some school districts even offer higher 
salaries for teachers in shortage areas. Such remedies are un~ 
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popular with teacher unions* which argue that al). teachers should 
receive the same base pay. Local businesses may be asked to pay 
bonuses to teachers in some communities in o^or to avoid con- 
frontations between the schools and unions on this issue. Given 
the magnitude of the $5*800-gap in starting salaries and the be- 
lief that many school districts and state and local governments 
are in a weak financial condition* Jt is not ciear .hat meaning- 
ful improvements can be expected in the salaries of teachers in 
science, mathematics* or other field.'. Lew salaries may continue 
to be a barrier to reducing the mathematics and science teacher 
shortage. 

K second major impediment to teacher recruitment concerns 
teacher morale problems. One component is the public's negative 
views of the public schools. The percentage of the public as- 
signing "bad grades" {C through F) for the public schools in- 
creased from 32 percent in 1974 to 52 percent in 1983.3? In 
1983* a quarter of those polled cited discipline as the top prcb- 
lem facing schools* followed by drugs at 18 percent* poor cur- 
riculum and standards at 14 percent* and lack o^proper financial 
support at 13 percent. 39 From the teacher's perspective* only 58 
percent of public school teachers polled by the National Educa- 
tion Association say they would become teachers again if they 
could go back to their college days. Teachers reported satis- 
faction with support they received from their principals and with 
the personal fulfillment of teaching. The main drawbacks cited 
were the amount of clerical duties and the lack of clerical 
help. 39 

Ernest Boyer* president of the Carnegie Foundation for the 
Advancement of Teaching* says that the recent Carnegie study on 
the American high school depicts the "loneliness and powerless- 
ness" of high school teachers. "The decline in the enthusiasm 
for teaching is rooted in the feeling that it's all been taken 
away — course putlines* textbooks* even the methods of supervi- 
sion and discipline . "40 Teacher morale problems thus include 
both problems of job conditions within the schools and the low 
public perception ' of the schools and of teaching. 

Improving teacher salaries and teacher morale could be ex- 
pected to improve both the quantity and quality of mathematics 
and science teacher^. These are probably relatively long-term 
goals. In the shorter run* the Congress is considering legisla- 
tion to, upgrade*mathematics and science education. The House 
^]5assed House of Representatives 1310* the Emergency Mathematics 
a, d Science Education and Jobs Act* on March 2* 1983- The 
Senate Committee on Labor and Human Resources reported another 
bill* Senate 1285* the Education for Economic Security Act* on 
May 16* 1983. As of this writing* the bill has not been debated 
by the Senate. The Senate bill authorizes $405 million in, fiscal 
year 1984 compared with $425 million in the House bill or about 
$1*900 per existing secondary school science and mathematics 
teacher. ^1 
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Table 2 



Mathematics and Science Teacher Quantity 
Remedies Included in Proposed Legislation , 

More new teachers for mathematics and science classrooms 

1. Programs to retrain teachers and other appropriate school 
personnel for mathematics/ science teaching . 

2. Partnerships where staff from businesses serve as teachers, 
"lecturers, consultants * to schools. 

3. Scholarships for preservice education for future mathematics 
and science teachers. 

4. Program to increase the representation of typically 
underrepresented groups in mathematics and science teaching. 

5. Development and dissemination of programs and materials for 
retraining teachers. 

Morale improvements to retain existing mathematics and science 
teachers 

1. Awards for teaching excellence.. 

2. Teacher service or employment in business firms under 
partnership programs. / 

3. Teacher incentive grants in which only graduates of training 
programs under this legislation can apply for grants of 
equipment and materials for their schools. 

Evaluation of the above programs 
SOURCE: H.R. 1310 and S. 1285. 



Table 2 shows the major teacher quantity remedies supported 
under these two bills. The need for additional teachers would 
be addressed by a variety of programs, including programs to re- 
train teachers from other fields, partnership programs where 
staff from businesses teach in the schools, and scholarship pro- 
grams at the undergraduate level- The proposed legislation also 
includes provisions such as awards for teaching excellence and 
summer jobs aimed at retaining existing mathematics and science 
teachers • 

yable 3 shows the major teacher quality remedies that would 
receive federal support. A variety of teacher^ training programs 
for current and newly employed mathematics and science teachers 
are included. The purchase of mathematics and science instruc* 
tional equipment and materials as well as the development and 
dissemination of materials and programs are proposed for support 
along with certain evaluation and research activities. 

29 
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Table 3 

Mathematics and Science Teacher Quality 
Remedies Included in Proposed Legislation 

Teacher training 

1. Assistance in the form of scholarships and traineeships for 
current or new mathematics and science teachers. 

2. Teacher institutes for mathematics and science teachers. 

3* Inservice training for teachers and other appropriate scHooL 
personnel. 

4. Rural area inservice training and curriculum development. 

5. Program to train teachers in programs operated by the 
private sector. 

Equipment, materials, and programs H 

1. Purchase of mathematics and science instructional equipment 
,and materials. 

2. Modernization or improvement of instructional programs, 

3. Development and dissemination of materials and programs for 
.training. 

4'; Partnership programs for the sharing of equipment and 
facilities. 

Evaluation and research 

1* Evaluation of the above programs. 

2. 'Research on effective methods of instruction, effective 

programs, curriculum development and materials, and teacher 
retention. . 

SOURCE: H.R. 1310 and S. 1265. 



The tables show only part of the multiplicity of strategies 
in the bills. Assistance is provided through State Education 
Agencies (SEA's), local education agencies (LEA's), and uhiversi 
ties as well ,as the various partnerships involving these institu 
tions and science museums and private^ businesses. In many cases 
— teacher training is a good example— the activity is authorized 
under separate component programs providing .funds to SEA's, uni- 
versities, and the like. 

la addition to remedies for both teacher quality and quan- 
tity, the legislation would support a variety of activities to 
improve school quality. Although school quality issues are 
outside the focus of this report, which is on remedies dealing 
with teachers and teaching materials and programs, we note for 
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completeness that the bills would support projects for gifted and 
talented students, . jcomputer learning and instruction, and foreign 
language instruction training and materials* The National Cam- 
mission on Excellence in Education and other groups have recom- 
mended numerous school quality reforms, Many are now being 
adopted or ^onsiderecj at the state and local level without any 
prospects for federal funding, For example* steps to lengthen' 
the school day and school year and proposals to require more 
mathematics and science courses for high school graduation may 
improve school quality by increasing the amount of learning 
time, Instituting a more demanding curriculum and tougher grading 
standards could be low cost locally initiated actions aimed at 
improving program quality, Some reformers call for sweeping 
long-term changes but even here some priorities such as teacher 
upgrading or reducing the teacher shortage seem needed^ 

SUMMARY 



Part of the concern for mathematics -and science education in 
the public schools seems powered by concerns over the position of 
the , United States in international economic competition^ We have 
seen that there are several educational priorities to improve 
American competitiveness, Some commentators recommend expanded 
and improved technical education while others do not, Those who 
place priority upon mathematics and science education are con- 
cerned over evidence of a shortage of teachers in the public 
schools, 



We find the data documenting these shortages— mostly opinion 
surveys— to be very weak and„believe teacher shortages to be basi- 
cally undocumented today, Concerns over the <Juality\of new math- 
ematics and' science teachers are largely inferential, based upon 
the prominence of uncertified teachers among the 9Vw recruits 
to mathematics and science teaching as well as tl£e\modest academic 
credentials of teachers generally, compared with bther professions 
requiring college study, In the next chapter, we examine one 
remedy to teaching quality concerns, institutes to upgrade exis- 
ting technical teachers, 
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CHAPTER 3 

WHAT ARE THE PROSPECTS FOR IMPROVING ~~ " "~ 

THE EFFECTIVENESS OF TEACHING THROUGH 
TRAINING TO UPGRADE EXISTING MATHEMATICS 
AND SCIENCE TEACHERS? 

t 

In the previous chapter, we noted that one of the major 

rfeTtt^diisarfor the "perceirTOt3^lT3^r~quai- i i :y af -tnat^hematl-es-and 

science teaching is a variety of in-servioe and summer training 
programs for existing teachers, Fears that too often teachers 
began their careers with less academic ability than might have 
been desired lead directly to suggestions for subject matter 
traini ng to up? r ^de_ existing teachers, 

A In this chapter, we explore one teacher quality improvement 

remedy—upgrading— in ordev to determine what the prospects are 
fpr improving the effectiveness of mathematics and science 
teaching through training to upgrade existing teachers, Wte 
begin with a review of the National Scienqe Foundation teacher 
institute programs, Which addressed very similar^ concerns from 
the mid-1950 1 s to the mid-1970 1 s, We examine tl^e types of 
institutes and scope of the programs and review evaluation data 
on the participants and some of the effects of these programs, 

" Tffe r ev iews-re search findin g s o n t h e- r e i^rt^-onsVrep- be tweeiv^the— * 

knowledge of teachers and the subsequent learning of their 
'students, We explore some more recent approaches to improving 
student learning and present somfe concluding observations* 

WHAT WAS THE EXTENT AND 
1 NATURE OF PARTICIPATION IN 
THE NSF INSTITUTE PROGRAM? 

Interest in the NSF program of institutes for precollege 
science and mathematics teachers has been revived foecause the 
institutes ftave again been proposed to help upgrade our tech- 
nical education capability, Although the current motivation is 
the improvement of our economic* productivity and international 
competitiveness, there is no indication that the role of the in- 
stitutes is conceived to be fundamentally different today than 
the program that "began with a summer institute at the University 
of Washington in 1954, 

Background ^ 

The original "goals of"tH^~NSF^ins"ti"tutes were- - 

"(1) to increase the effectiveness of teachers by 
broadening and updating their scientific 
background; " * , , 




(2) 



renew interest in an attitude of teachers toward 
science and their task as a factor in the 
motivation and encouragement of their students* 



(3) improve communications, sympathy and understand- 
ing between groups (researchers versus teachers, 
for example) * 

Three types of institutes were developedt summer institutes, 
in-service institutes* and academic year institutes, Summer 
institutes, the first type funded, were patterned after indus- 
trial prototypes* developed in the 1940's by General Electric, 
Westinghouse, and others, ^ Summer institutes, typically hel^d at 
college campuses, featured small, subsistence stipends for most 
of those teachers attending, Lasting from 6 to 8 weeks* summer 
institutes would aim to improve the mastery of subject matter in 
mathematics and a variety of science fields, 4- Table 4 shows 
-that^summer^ institutes -were— the largest o^-the^igstitute pro- , 
grams, measured by number of grants, number of -participants, 
or dollars, {Dollars in table 4 are unadjusted,) f 

The next largest in terms of number of grants or partici- 
pants were the in-service institutes, Providing training on a 
part-time basis during the school year, the in-service institute 



T able _4 

NSF Institutes for Secondary School Teachers* 
Number ofTGrants* Number of participants. Amount Qblfgated, and 
Cost pe r Participant* Cumulative for Fiscal Years 1954 through I973t 





Total 


and by Type of 


Institute 




* 




Number 


Number of 


Amount 




Cost per 




of qrants % 


participants % 


obligated 


% 


participant 3 




U) 


(21 


(3) 




(4) 


Summer institutes 


6*084 57 


278,629 58 


$330,705,337 


64 


$1* 187 


In-service 


3,817 36 


180*053 38 


47*630,290 


9 


265 


institutes 












Academic year 


767 7 


21,453 4 


135,717*737 


26 


."6 * 326 * 


institutes 










?1*071 


Totai 


10*668 100 


480*135 100 


$514,053,364 


99 


a Column 3 divided 


by column 2, 


t 









♦SOURCE: National Science Foundation* Frecollege Science Curriculum Activities 

of the National Science Foundation: A Report of the Science Curriculum 
Review Team (Washington, dTcTI National Science Foundation* 1973), 
p, 25. * 4 
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program paid for a teacher's tuition, local travel allowance, 
and textbook fees. Groups of teachers, typically 40 to 45 in 
number, would meet after school hours or on Saturdays. 5 

The academic year institute program consisted of full-time 
programs of study, often leading to a master's degree, The pro- 
gram objectives included updating Subject matter knowledge, 
training in depth, and advanced specialized training. ^ Later, 
even the summer .institutes were sometimes organized on a sequen- 
tial basis for high school teachers to obtain advanced degrees.' 
Academic year institutes were like sabbatical fellowships with 
specially designed curricula and some courses not normally of- 
fered by the^universijiy . Using table 4 to obtain crude cost per 
participant figures, note that academic year institutes cost 
$6,326 per participant. The cost may be attributable to 
stipends made available to provide partial salary recovery.? 

At the same time, NSF began work (to upgrade obsolete 
mathematics and science curricula. O^er the years, an "alphabet 
curricula** ranging frOm School Mathematics Study Group (SMSG) 
mathematics to Biological Science Curriculum Study (BSCS) biology 
were developed. A separate program, tAe Cooperative College- 
School Science Program had been set up ^o provide collaborative 
programs between secondary schools and colleges for the improvey 
, inent of school science instruction. 8 Th^s program was used by 
1964 to support close collaboration between college and secon- 
dary school officials in **the planning, adaptation, and intro- 
duction of ^he newly developed science curricula into one or more 
nearby school systems. M ^ This was an ealriy step by NSF into 
curriculum implementation activities • !0 j As the curriculum 
materials became available, institutes w^ere also utilized to help 
teachers learn the curricula. By 1965, about 20 percent of the 
summer institutes' had a major orientation towards' one of the 
revised curricula. 11 Subsequent to 1970, NSF phased out the 
discipline-oriented institutes and emphasized curriculum 
instruction for teachers and the implementation of curricula in 
specific schools or school districts.! 2 ' By* fiscal year 1974. the 
institutes had, been replaced with implementation workshops . " * 
The Office of Management and Budget apparently was opposing NSF 
programs it considered to be continuing federal aid and was more 
favorable toward problemsotving projects such as helping some 
region upgrade science education in its local schools- 1 ^ The 
curricula activities had become a major dilemma for NSF. On the 
one hand, the slow rate of adoption of NSF-funded curricula by 
school districts created an incehtive for NSF to assist the 
implementation effort. On the other hand, the charge that the 
federal government was influencing curriculum content stirred 
controversy in many quartets. !5 

A variety of causes led to the termination of the imple- 
mentation workshops in 1975. First, the shortage of scientists 
had turned into a surplus Second, the director of NSF 
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declared victory- stating that the institutes had achieved their 
maximum benefit. 1? Third, a major dispute had erupted over the 
cultural relativism portrayed in a social science curriculum, 
"Man — A Course of Study," funded by NSF. The dispute expanded 
into larger questions of NSF decisionmaking and marketing 
questions. 18 Fourth,* the NSF precollege activities were a 
concern of its governing board, which was disturbed over growing 
problems of the limited funding available for university 
research. ^ Fifth, the opposition of the Office or Management 
and Budget to continuing aid programs such as the institutes was 
noted in the previous paragraph. 

In fiscal year 1977, the Congress reinstated a small 
precollege teacher development program. This continuing program 
has been similar to .the former in-service institute program, 
part-time and local in itfc orientation, with* more input from the 
teachers and consequently more emphasis on classroom problems 
than under the institutes. NSF has funded no evaluations of 
this program. 

There have been several other small NSF programs aimed at 
having impacts on mathematics and science education in elementary 
and secondary schools. An information dissemination for science 
education program aimed at disseminating information on 
curriculum and instructional materials and otherwise improving 
instruction was funded through fiscal year 1980. A development 
in science education program has- funded the development and 
experimentation with ideas having the potential for improving 
science education. The Research in Science Education program is 
aimed at creating an organized body of knowledge on science 
education. 

Niew programs totaling $15 million in fiscal year 1983 and 
$54.7 million in 1984 include a materials development for 
precollege science and mathematics program. ^0 The research and 
development activities under this program are similar to those 
discussed abovj. A second new effort is the Presidential Awards 
for Teaching Excellence in Science and Mathematics program. In 
1983, one science and one mathematics teacher from each state 
was invited to a ceremony in the District of Columbia. Under 
this program, each teacher received a $5,000 grant for use by his 
or her school in science and mathematics improvement. Another 
new program is the Honors Workshops for Teachers of Science and 
Mathematics, a highly selective search for outstanding teachers. 
The teachers will attend workshops for about 4 weeks and may 
introduce innovations from the workshops into their schools upon 
their return. 

In some respects, the current goals seem closer to the 
earlier than to the later NSF institute programs. These last 
institutes marked a time of reduced concern over the quality of 
teachers and a growing internal problem over increasing the r 
adoption or utilization of the ftSF-funded curricula. 
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Evaluation criteria 

•We first explore the most elementary measure of the 
institute program's effectiveness, the extent^ of institute 
participation. After reviewing the extent of participation we 
examine the characteristics of participants. We then turn to 
another measure of impact, the effect of institute attendance 
upon the subsequent science or mathematics achievement of the 
attending teachers 1 students. (Other possible evaluation 
criteria are discussed in appendix I.) 

Extent of participation * 

As "the figures in—table 4 may have- suggested, participation 
in the NSF institute programs was'widespread. A study of mathe- 
matics and science teachers reported in 1962 that 36 percent of 
the public senior high school teachers and 24 percent of the 
junior high school teachers had attended NSF institutes* While 
limited to the prior 5 years,- in effect this is a study o f 
participation rates early in the NSF program. 2Jk* 

In a 1971 survey of 2,489 public secondary schools, 51 per- 
cent of secondary school science teachers reported having 
attended one or more *NSF summer institutes. Many people 
attended more than one institute; 34 percent of the science 
teachers had attended two or more NSF summer institutes. 
Furthermore, 28 percent of the science teachers reported having 
attended NSF in-service training and 9 percent attended NSF 
academic year institutes* 22 A^weakness of this survey, however, 
was its low rate of usable respbnses^only aE>out~39 percent tcrtT" 
teachers. 

A 1977 national survey of teachers at all grade levels 
asked teachers about their attendance at NSF institutes* As 
table 5 shows, at grades 10-12, 52 percent of the science 
te/ichers and 38 percent of the mathematics teachers reported 
attending one or more institutes* At grades 7-*9, the attendance 
estimates drop to 33 percent of science teachers and 27 percent 
of mathematics teachers* The figures are lower in the 
elementary grades where teachers usually are not specialists but 
teach all academic subjects. The teacher response rate of 76 
percent in this survey apparently is the response from 
participating schools r thus a participation rate based on the 
total sample of schools, which was requested to participate^ 
would be lower* 23 

Using questionnaires to ask teachers to recall past in- 
stitute attendance as well as sponsorship of these institutes is 
likely to produce substantial measurement error« Major dif- 
ferences in questions asked, year of administration, and survey 
design also make it difficult to compare these r,«:r^eys. How- 
ever, all three surveys suggest that participation in the NSF 
institute programs was fSlrly extensive; about half of the 
senior high school mathematics and science teachers participated 
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Table 5 



Percent of Teachers Reporting Attending 

~- One^-or More-NSF—insti-tuteSy-by - — 

Grade Level and Subject Area, 1977 



Grade and subject 



Grades 10-12 

Science 

Mathematics 
. Grades! XrSL 

Science 

Mathematics 
Grades 4-6 

Science 

Mathematics 
Grad e s K ^ 3 - 

Science 



Percent of teachers attending* 



52 
38 

33 
27 

13 
6 



Mathematics 



(/ 



2 
5 



a Allocates the missing or inconsistent responses proportionately 
to the attending and nonattending categories. 

SOURCE* Iris R. Weiss*, 'Report of the 1977 National Survey of 
* Science, Mathematics and Social Studies Educatio n 

f Research Triangle Park, N.C.; R esearch Triangle 



Institute, 1978), p. 69. 



in the institutes. Clearly the elementary school teacher 
participation was lower, estimated at 14 percent in another 
study. 24 This is consistent with our knowledge of NSF 
priorities. In the" absence of departmentalization in the 
typical elementary school, NSF emphasized training key teachers 
or supervisors who might in turn improve the mathematics and 
science instruction in their home schools. This attempt at 
having an impact on the large number of elementary schools and 
elementary school teachers was not considered successful. 
Elementary school teacher summer and in-service programs were 
terminated in fiscal year 1966. 2 ^ 

Characteristic of participants 

Who — in the NSF institute experience — applied and • 
^subseguently^waa^adii^ NSF in sti t utes? The 

process was similar to standard college admissions. Teachers 
applied individually and were^accepted or rejected by institute 
personnel * A common^appiijjtftion form was used but NSF did not 
specify to institute ctlrectors how they were to use the 
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information provided. The institutes in turn had to specify in 
their proposals how they would select- participants, but once tfie 
applications were sent in there were no selection rules from 
NSF. The NSF position was simply that the major criterion * , 
should be T the ability to benefit from an institute. Several 
studies are available, which suggest the characteristics of 
those who participated. 

A 1962 study describes a sample of 1,845 target - 
teachers — teachers who taught mathematics or science at least 40 
percent of the time and were from 25 to 55 years old — in 427 
participating secondary schools. ~ Those teachers reporting that* 
they applied to NSF summer, in-service, and academic year 
inst.it.utes, as compared with nonapplicants 

—had more college qredit hours in mathematics or 

science at both the undergraduate and graduate levels; 

— reported more job satisfaction; v 

-•-engaged in more professional activities, including 
reading journals and belonging to and holding office 
in professional societies. ^ 

The study characterized nonapplicants as having generally low 
levels of self-improvement motivation in their work. They' 
recognized that they needed better subject matter preparation 
but did not apply to attend institutes. Although they reported 
keeping up through reading, in fact they read fewer journals 
than ^applicants. The nonapplicants were overrepresented in 
rural areas and small schools having less extensive course 
offerings. These teachers thus were much more likely to teach 
in at least one other subject matter field, which presumably 
decreased their identification with mathematics or science. We 
caution the reader that we were not able to verify that the 
findings reported here were consistent with the data collected 
due to the limitations in the statistical tables provided. 

This study was disturbing in the sense that it suggested 
that the large, number of nonpar tic ipants in institutes could 
benefit from additional preparation but seemed unlikely to seek 
it. We discuss training for less qualified teachers at the end 
of this chapter and in chapter 6. 

A 1971 survey identified teacher and school characteris- 
tics, vhich were associated with participation in an institute. 
Regression analyses aimed at relating institute attendance with 
teacher and school characteristics yielded few notable 
relationships. In analyses of the total sample of about 2,193 
teachers, only the teachers* number of semester hours in college 
science courses wais significantly related to attendance at a 
summer .institute. No teacher or school characteristic was found 
to be significantly relate3 to the 4 amount of participation in 
in-,service institutes. Attendance in any NSF institute (as. 




opposed to the analyses for different types of institutes 
individually) was related to the number of semester hours in 
college science and years of secondary school sciew teaching 
experience. 27 This analysis — unless the choice of dependent 
variables was unfortunate — suggests that there was little to 
differentiate those who attended few or many NSF institutes. 

Several studies comparing those accepted and rejected for 
institute .attendance have utilized data from the standard NSF 
institute application form. Table 6 provides data from a syn^ 
thesis paper comparing summer institute selection studies in 



Table 6 



Characteristics of Teachers Accepted and Rejected 
for MSF Summer Institutes* by Year 



1957 1960 1964 

Accepted Rejected Accepted Rejected Accepted Rejected 

Mean number of 13.2 11.6 m 9.4 8.9 8.8 8.2 

undergraduate 
semester hours 
in science 



Mean number of 1.6 1.0 1.5 1.5 l.t> 1.1 

graduate semes- 
ter hours in ' 
science 

Mean undergrad- 2.9 - 2.7 2.7 2.6 2.6 2*4 

uate grade point 
average in math- 
ematics courses 

Percentage of 64.9 56.9 70.3 64.0 47.3 42.6 

undergraduate 
science or math- 
ematics majors 

Mean number of 
years teaching 
experience for 
selected 
subjects 



Mathematics 


5.1 


5.3 


4.1 


3.8 


Physics 


2.2 


2.0 


1.2 


0.9 


Chemistry 


2.6 


2.2 


1.6 


1.2 


Earth science 


0.2 


0.3 


0.4 


0.4 



SOURCE? Adapted from Raymond K. Berger and Frances R. Berger* A 

Study o^ the Attributes of Applicants to National Science 

Foundat+Cn Summer Institutes in 1964 (Los Angeles: 

Psychometrics Consultants* 1965), pp. 7-31. 

* ■ 
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1957, 1960r and 1964. Some trends (not all of which are statis- 
tically significant) are that high school teachers selected had 



— more semester hours in science than those rejected; 

— more graduate semester hours in science (except in 1960 
when there^was no difference); 

— slightly higher undergraduate grade point averages; 

— greater likelihood of majoring in science or mathematics 
at the undergraduate level; ^9 

— more years of science and mathematics teaching experience 
(except mathematics in I960 and earth science)* 



In short, those with stronger" academic credentials ,and more 
background in science and mathematics had higher probabilities 
of, selection. There was some evidence of more memberships in 
professional organizations among those accepted. *9 

Table 6 data also suggest that by 1964 the qualifications 
of institute participants declined somewhat. Those accepted had 
8.8 semester hours of undergraduate science compared with 9.4 in 
1960 and 13.2 in 1957. A smaller percentage of the 1964 accep- 
tees had been undergraduate science or mathematics majors than 
in the prior years* Efow^yer, those selected continued to have 
somewhat stronger academic preparation than those rejected. 
Table 6 also shows that by 1964 those accepted for institute 
attendance had an average of less than 2 years experience 
teaching physics or chemistry (and earth science teachers had 
even less experience). There was some concern in NSF that the 
availability of institutes almost upon graduation from college 
might be undercutting the quality of teacher preparation pro- 
grams* These statistics ori'science coursework and teaching 
experience support the 1970 NSF position we cited earlier — that 
the institutes had achieved one of their goals* 

In summary, in this review of completed studies we have " 
found some common factors. Participants had taore undergraduate^ 
college science in all studies. Participants had nrore y ears 
secondary school science teaching experience^ than nonpar"tttfi'-"~* " 
pants except for the 1962 study* Some "professionalism* 
findings were found in the 1962 and 1964 studies* J 

»- 

The institute program practiced a degree of academic selec- 
tion or creaming* While the ac^demic^ differences between the 
average participant and nonpar ticipartt were apparently riiodest, 
we found them consistently. Since tpe' institute prograirs were 
run by colleges and universities, it is not surprising that they 
operated on essentially a graduate school model of selecting the 
best academically qualified applicants* It seems unlikely that 
NSF resisted this practice* The academic- university orientation 
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of NSF with its continued reluctance to conduct activities in 
precollege education has been cited widely. 30 

In conclusion* this subsection has shown that participation 
in the NSF institute programs was widespread. Participants had 
somewhat stronger academic qualifications — and possibly greater 
prof ess ional ism- -than nonpartici pants . We saw some evidence 
that over time the institutes remained selective but lowered 
requirements beioW those of earlier years. The question of how 
to attract and subsequently upgrade the nonapplicant teachers in 
programs scch as these remains a troubling issue. 

HOW EFFECTIVE WERE THE NSF 
INSTITUTES IN IMPROVING 
STUDENT ACHIEVEMENT? 

We examine the impact of teachers* attendance at NSF 
institutes upon the subsequent achievement of their students ^in 
mathematics or science. In appendix I, we cite the other major 
outcomes of institute attendance that have been studied and the 
advantage of student achievement over the alternative outcomes. 
A major value of student achievement is that it represents the 
most fundamental goal of our educational system and, directly or 
indirectly, of most education programs at the precollege level. 
In addition, student achievement was a central concern of 
education commission reports that were published in 1983. The 
disadvantages of student achievement measures are technical 
considerations, which we c^te in appendix I. 

Applying our criteria for consideration and selection of 
studies (see appendix I), wo found* pnly one qualifying study of 
the impact of NSF institute atttfrtQance upon student achievement. 
However, it included four samples* so it is really one report on 
four studies. 

This effort — reported by Victor Willson and Antoine 
Garibaldi in 1976 — examined >the achievement of students whose 
teachers had participated in NSF institutes compared with the 
achievement of students whose teachers had not participated. 31 
Principals of' sampled schools randomly selected one science or 
mathematics teacher who in turn was asked to complete a 
questionnaire, take the applicable National Teacher Examination, 
and select one class at random" for testing. Science students 
took the Test of Achievement in Science (consisting of 40 items 
selected from the National Assessment of Educational Progress) 
while mathematics students took the Mathematics Achievement Test 
{composed of 40 items fpom the National Longitudinal Study of 
"Mathematical AbiTitiesOf. ~^Two~"fi5rms "Were developed for each, one 
for 8th and one for lltjn graders. A total of 346 teacher^ and 



classes participated 



classes were from schools sampled in Wyoming, South Dakota, and 



Mississippi regions whi 
and Indiana regions. 



science and 211 in mathematics. Science 



le mathematics classes were in California 
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The results, which are discussed in more detail in appendix 
II, show that NSF institute participation was a statistically 
significant factor in high school science and mathematics 
achievement* The statistical significance levels were *06 and 
.02, respectively. However, institute participation was not 
significant for 8th-grade student achievement in either science 
or mathematics. 

The results of four samples are not enough to conclude that 
NSF high school teacher institutes are effective and that junior 
high school teacher institutes are not. 32 we searched, 
therefore, for related evidence. Our search focused on the fact 
that subject matter training was a major element of the NSF 
institutes. 33 Does more general research on the relationship 
between teachers 1 knowledge and the knowledge of their students 
make a plausible case for expecting that the NSF institutes 
would help improve student achievement? 

ARE MORE KNOWLEDGEABLE 

TEACHERS MORE EFFECTIVE TEACHERS? 

What makes an effective teacher? If a teacher's primary 
function is to transmit knowledge, it would seem that the amount 
of knowledge the teacher has in the subject or more globally 
would be critical. Yet most of us have endured studying under 
brilliant teachers who were inept communicators. A teacher 
should also transmit ,some of the fascination' with his or her 
subject to help motivate learning. Personality and pedagogical 
technique are other factors in the complex determination of t 
effective teaching. - * 

A 1963 review of studies on the relationship between 
teachers' intelligence and effectiveness showed little evidence 
of an association in the then-current literature. ^ The in- 
fluential Coleman report concluded in 1966 that teachers* vocab- 
ulary test scores were associated, with student verbal achieve- 
ment for all minorities but no teacher or other school factors 
accounted for much variation in student achievement. The 
analysis found that "schools bring little influence to bear on a 
child's achievement that is independent of his background and 
general social context. "^5 

In 1969, Arthur Rothman and colleagues published results of 
three studies of the relationship between teachers 1 tested 
knowledge of physics and their students* subsequent learning 
in physics. The three studies showed little relationship: 
one found three out of four correlations to be small but 
-positive, one showed no overall relation between a large set- of— 
teacher and student variables, which led the investigators to 
reject examination of individual bivariate relationships, and one 
produced a small negative relationship between teacher 
achievement and student learning. 36 
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A 1972 report by Edward Begle found in a study of 308 
algebra teachers that teacher understanding of basic algebra had 
a significant; positive correlation with student achievement in 
the understanding of 9th-grade algebra. However, Begle concluded 
that "while this correlation is statistically significant, it is 
so small as to be educationally insignif icant* "3' In three other 
teacher-student knowledge correlatigns, there were no significant 
relationships * 

In 1975, Frances Lawrenz reported from a study of 236 
secondary school science teachers that teacher knowledge of the 
processes of science was positively related to student achieve- 
ment but teacher subject matter knowledge as measured by the 
National Teacher Examination was negatively related to student 
achievement* Willson and .Garibaldi reported the absence of 
relationships between teachers' science (or mathematics) ability 
and their students' achievement, as we note in the detailed 
discussion of that research in appendix 11*38 

Finally, Theodore Eisenberg replicated the Begle study to 
determine if the NSF institute participant volunteers Begle used 
may have unduly limited the variation in teacher knowledge* In 
his study of 28 Columbus, Ohio, junior high school teachers, 
Eisenberg reported in 1977 that he found nonsignificant correla- 
tions between residual teacher effects on student achievement {in 
algebraic concepts and algebraic skills) and teacher knowledge. 
The small sample size is a particularly serious weakness of this 
study in that it makes it diffiqult to produce significant 
correlations* Eisenberg reported that six other studies {not in- 
cluded here) were also in accord with Begle' s findings* 39 

As a group, these studies fail tQ show any relationship 
between teacher knowledge and the knowledge gain of their 
students* * Several of the studies are small and subject to 
methodological criticisms such as having a possibly biased group 
of participating teachers* However, another recent synthesis 
included dissertations and also found no consistent relationship 
between the knowledge of teachers and their students' achieve- 
ment, suggesting that the exclusion of dissertations has not 
biased our findings *40 Furthermore, this finding is consistent 
with the direction -of - schoat-re search in- the—las t~-de cade— As 
Purkey and Smith argued in a recent review article, 

v "the general finding [is3 that easily measurable dif- 
ferences among schools (class size variation from 20 to 30 
pupils, existing differences in teacher preservice training, 
teacher experience and salaries * * *) have little consist- 
erit relationship to student achievement * "41 

Teacher knowledge seems to be one such variable that "can be 
measured and, in theory, changed relatively easily" through 
spending more to hire more knowledgeable teachers*42 More 
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recent studies have looked at different variables and not the 
teacher knowledge and related variables examined here. 43 

We cannot say that there is no link between teachers 1 
knowledge and their students 1 learning with the confidence a 
researcher can have in saying there is no link between treating 
with Laetrile and curing cancer. However, research to date 
clearly has failed to show a straightforward relationship 
between teachers' knowledge and the subsequent learning by their 
students in mathematics' and science, at least for teachers in 
classrooms in the early 1970' s. This finding challenges a com- 
mon assumption — the belief that the more a teacher knows about, 
the subject being taugh^, the better the teaching that will be 
done and the more the student will learn about the subject in 
question. * 

It may be that there is evidence of favorable effects of 
higher levels of teacher knowledge on encouraging future careers 
in science and mathematics or some of the other outcomes we cite 
in appendix I. We have not reviewed this literature. Regarding 
the impact of teachers' knowledge on the important outcome of 
student achievement, we find no consistent relationship. We 
discuss some of the implications of this research for teachers 
in classrooms today at the end of this chapter. 

WHAT DOES RECENT CLASSROOM RESEARCH TELL "US 
ABOUT HOW TO IMPROVE STUDENT ACHIEVEMENT? 

We have examined evaluations on the effectiveness of NSF 
institutes in improving student achievement and found only one 
study of sufficient quality . Since many of the NSF institutes 
* focused on upgrading teachets' discipline or content 'knowledge, " 
we then turned to more general research on the relationship be- 
tween teachers' knowledge and the achievement of their 
students. The results of this latter inquiry failed to suggest 
benefits in upgrading teacher knowledge. We now turn to more 
recent approaches of attempts to improve teaching effectiveness 
through understanding and changing the classroom behavior of 
teachers. Rather than trying to increase teachers' knowledge of 
science or mathematics, this approach has the goal of improving 
how the teacher manages learning in the classroom. 

Researchers in the 1970 's identified teachers who were ef- 
fective in increasing student achievement, observed them, as 
well as less effective teachers, and devised objective ways to 
classify and count specific teacher behaviors. It^was then 
possible to study a cross-section of teachers and correlate 
objectively measured teacher classroom behavior with the amount 
of student learning. This is called "process-product" research 
because it relates the classroom processes to the outcome or 
product produced. The result of this research is lists of de- 
sirable behaviors, which turn out to include both classroom 
management skills and instructional strategies. Effective 
teachers are more likely than less effective teachers to 
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— provide and enforce clear and consistent rules of conduct; 

— instruct and drill students in well thought out procedures 
for classroom behavior; 

— respond more promptly to incipient misbehavior* 

— maintain well-paced momentum in instruction* 

— assign independent work* which will provide students with 
high (90-100 percent) levels of success; 

— interact frequently with students providing prompt and 
frequent academic feedback to students by correcting 
papers* answering questions. 44 

How confident can we be in this list? We found no 
synthesis or assessment of the technical quality of this 
research and such an undertaking was beyond the scope of this 
report* The substantial number 4 of publication's in this area is 
somewhat offset by the fact that it represents a prolific "bujtput 
by a relatively small group of researchers* While classroom 
research may "still be in its infancy 11 as one of Ijts leading 
members has written* it is encouraging that there is some 
consistency in* the findings in*the sense of pointing toward more 
active and organized teaching. 45 Th e research has a major 
weakrvess for the purposes of this report in that it is limited 
to the elementary grades while our analysis focuses on secondary * 
schooling. s 

The process-product researchers have extended their 
research to develop training or instructional packages v/ith the 
goal of training teachers to otilize those techniques* which 
their research has found to be effective. The researchers; have 
then studied the training programs with two objectives* First* 
they have sought to determine the extent to which teachers im- 
plemented the changes recommended by the training. Second* they 
studied the success in classrooms whexe teachers had implemented 
the training in terms of improved student achievement. Put 
another way* they sought to determine if classroom management 
skills could be taught and if the success of teachers who uti**" 
lized these methods naturally could be replicated. The results 
were encouraging in studies of 4th-grade mathematics in 
Oklahoma* lst-grade reading in Texas* and 3rd-grade reading in 
California*^ The fact that training programs were relatively 
short and required few materials is encouraging. Programs we 
identified ranged from two sessions to seven workshops. 



We find the work of the process-product researchers encour- 
aging but not too useful for our purposes. The extension of 
this approach to exploration -of eff^tive teaching in secondary 
school mathematics and science classes could be a valuable 
contribution. 
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We end this section on process-product research by noting 
that otJier unrelated research, while beyond the scope of this 
report, may be of inte:r«t to some readers. One set of research 
concerns the literature on the nature of student understanding 
of science and mathematics concepts. 4 ^ There is also an emerg- 
ing literature on the use of computers to teach science and 
mathematics concepts through simulation.^ 

CONCLUSIONS 

This chapter raises questions about approaches that have 
substantial teacher training components. Such programs are a 
prominent part of proposed federal legislation.^ Our analysis 
suggests that programs geared at upgrading existing mathematics 1 
and science teachers may not produce results in terms of improved 

student achievement. 

* * * 

We reviewed NSF institute participation data and found that 
this earlier program to increase teacher quality attracted 
roughly half of the senior high school mathematics and science 
teachers who were in the classrooms during the 1960*8. Many 
teachers took extensive coursework under' the Academic Year 
Institutes and later even, obtained master's degrees under 
sequential Summer Institute programs! *The participating half of 
the teaching force was somewhat more academically qualified than 
those not applying or nQt accepted. While the qualifications 
of 'participating teaphers declined somewhat over time, the 
qualifications of -applicants in general apparently declined as 
well, leaving a modest gap between those accepted and rejected. 
How to reach the nonparticipating teacher was never resolved 
under the NSF program. Perhaps the closer ties between 
universities and school districts under the proposed legislation 
would give school dist-rists more leverage to obtain broader 
teacher participation. Under 1 the NSF institute program, teach- 
ers applied at their own initiative. 

We examined the effectiveness of NSF institutes in increas- 
ing the science and mathematics knowledge of students of the 
participating teachers but found only one study that met our 
minimum criteria. It found mixed results, with evidence of 
effectiveness o£ the institutes (statistically significant' 
differences) at the senior high school but not the junior high 
school level • 

We then searched for related evidence, focusing on the fact 
that subject matter training was a major element of the N£&* in- ' 
stitut^s. We looked at the general research on the relationship 
between teachers* knowledge and the Knowledge of their students 
as a way of further assessing the effectiveness of the strategy 
of upgrading tpacher subject matter knowledge. .The studies, of 
secondary school science and mathematics learning in the 1960's 
and 1970 's failed to show any straightforward relationship 
between teachers 1 knowledge and subsequent, student learning. A 



We then turned to more recent approaches to increasing 
teacher effectiveness — the process-product research. This 
effort focuses on classroom management rather than on teacher 
knowledge, \nd it is aimed at identifying objectively the 
components of effective teaching, training teachers to adopt 
those components in their own classrooms, arid attempting to. 
replicatev^e effectiveness results for the trained teachers' 
classes. The results of this emerging research area are* 
promising, but process-product research has not yet been 
attempted at the secondary school level. 

* * * 

This review of teacher quality remedies is not encourag- 
ing< Perhaps if we, had included an examination of school qual- 
ity improvement remedies cited in chapter 2 or school management 
approaches, we would have foynd different results. 

There must be some limits on the lack of relationship 
between teachers' knowledge and subsequent learning by their 
students. Someone completely ignorant of calculus presumably 
could not teach it. The findings we reported were 'based on the 
range of knowledge of actual teachers in classrooms, which pre- 
sumably filtered out those with very little knowledge of the 
subject. Perhaps, the recent appearance of significant numbers 
and percentages of uncertified new mathematics and science 
teachers in the schools may have significantly lowered the 
average knowledge level and could therefore lead to different 
findings than those reported here. From a policy perspective, 
this may suggest a training approach of targeting training on 
the least qualified -mathematics and science teachers. This 
approach in some ways is the opposite to the NSF institute 
strategy, but it seems to be called for in view of the apparent 
needs combined with the unlikely effectiveness of training for 
other groups of teachers. 

This examination of effective approaches to increasing 1 
'teacher quality irv mathematics and science education has been 
hindered by the shortage of appropriate evaluation and research 
studies. We, return to this matter in chapter 5 where we out- 
line some possible approaches to evaluation under new proposed 
legislation. Now we turn to the issue of improving the quan- 
tity "of science and mathematics teachers through retraining 
programs. f 
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CHAPTER 4 / 

HOW VIABLE A SOLUTION TO 

SHORTAGES OF MATHEMATICS AND SCIENCE TEACHERS 

IS RETRAINING TEACHERS CERTIFIED IN OTHER FIELDS? 

The purpose of this chapter is to examine the feasibility 
of retraining teachers certified in other fieldo to become mathe- 
matics and science teachers. Retraining teachers already certi- 
fied in other fields is one of several possible ways of reducing 
or eliminating a shortage of mathematics and science teachers. 
If, in fact, teacher knowledge is less important for student 
achievement than has been assumed as we suggested in chapter 3, 
then .the orderly movement of teachers from other fields into 
mathematics and science teaching is a realistic possibility. 

Another possible solution to teacher shortages that focuses 
on retraining is training mathematicians and scientists working 
in or retired from positions in industry. These persons would 
already possess adequate knowledge in the subject area but would 
need to be trained as teachers. There are other possible 
solutions to the shortage problem, such as raising the pay 
for mathematics and science teachers, waiving state-certification 
requirements, increasing the use of emergency credentialing, 
increasing the size of mathematics and science classes, and 
multiplying the teacher's "reach" through the use of audiovisual 
aids o , computer-based instruction. These options are summarized 
elsewhere.^ 

In this chapter, we discuss the development of teacher re- 
training programs and the means we used to identify and analyze 
existing programs. We discuss the types of programs we* identi- 
fied and examine their varying approaches to runaing, candidate 
selection, and program length. We discuss the criteria by which 
the effectiveness of retraining programs may be evaluated and 
the extent to which these criteria can be applied to these pro- 
grams. Finally, we draw some conclusions from the analysis. 

WHAT ARE THE CHARACTERISTICS OF THE NEW 
RETRAINING PROGRAMS? 

Development of retraining programs 

Programs to retrain teachers certified in other subjects to 
become mathematics and science teachers have developed in re- 
sponse to the shortages of qualified mathematics and science 
teachers combined with a surplus of teachers in some other 
fields. Enrollments of children have declined in many school 
districts around the country in recent years, and reductions in 
force of teachers and reassignments to other subject areas have 
sometimes occurred. (The American Federation o? Teachers 
estimates that 140,000 teachers havu been affected by reductions 
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in force during the period 1981-83*) In some districts* 
particularly those with unxon contracts* seniority often 
determines which teachers keep their jobs wh*n there is an 
oversupply. Retraining nay be a more appealing option 
than to release some teachers and hire others* Retraining also 
appeals to those persons who believe that the pedagogical skiils 
of an experienced* dedicated teacher may be more important than 
content knowledge (see chapter 3)* especially where the aim o£ a 
retraining program may be to prepare participants to teach basic 
lather than advanced courses* The major objection to retraining 
X>rograins is a concern that retrained teachers will lack a depth 
of knowledge in mathematics or science needed to teach 
effectively, 

Study design and sample selection 

We examined literature on mathematics and science retraining 
programs Cor teachers and discovered that the literature did not 
discuss the effectiveness of mathematics and science retraining 
programs (presumably because such programs have onl y recently 
been developed)* but it consisted primarily of discussions of the 
problem and possible solutiors* The most useful "literature" for 
this review consisted of recent speeches or press interviews, 
position papers* descriptive brochures* and interoffice memoranda 
that we often obtained directly from those affiliated with 
retraining programs* 

We used a variation of "snowball sampling" to select 
programs to review* We used snowball sampling because we were 
faced with the perplexing methodological problem of how to 
identify the national population of mathematics and science 
teacher retraining programs* Snowball sampling uses probability 
methods to select the initial respondents for a survey and then 
obtains additional respondents from information provided by the 
initial respondents * 2 our particular variation of snowball 
sampling consisted of conducting a telephone interview with an 
official of each of the existing state-level mathematics and 
science retraining programs that had been identified in an 
Education Week article listing such programs and obtaining 
additional respondents from the initial respondents; that is, we 
selected each initial respondent with a probability of one and 
defined existing state-level programs as programs that were in 
operation or had been funded. ^ The initial state-level 
respondents referred us to other states that had recently passed 
legislation covering mathematics and science retraining programs 
and to school districts and universities that were conducting or 
planning to conduct such programs* While this methodology does 
not guarantee comprehensive coverage of all programs or provide a 
sample systematically chosen from a predefined universe as would 
more traditional methods* we believe sufficient programs have 
been identified to discover patterns of commonality and to 
suggest tentative conclusions, which can be tested as more data 
on retraining programs become available* 
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We identified 17 mathematics and/or science retraining 
programs and interviewed the program managers over the telephone 
concerning the progress of their projects. The interviews 
covered the goals of the retraining programs, the procedures used 
to select participants, the content and length of the 
programs, evidence of the effectiveness of the programs, and the 
cost of the programs. We also interviewed officials over the 
telephone from professional organizations knowledgeable about 
mathematics or science teaching (National Council of Teachers of 
Mathematics, National Sci ence Teachers Assoc lat ion , and National 
Education Association). 

Types of programs 

For the purposes of this analysis, we classify each program 
as either a SEA, a LEA, or a university program. In all pro- 
grams, participating teachers are enrolled in classes aimed at 
retraining them to teach science or mathematics subjects. 
A SEA or LEA program is defined as an organized system at the 

—state- or— local- leveL-to. provide f u n d i n g— a ncL-Oth er^auppoxt _JLo_ 

identify, recruit, and provide training to teachers of other 
subjects to assist them to be certified as teachers of 
mathematics or science. Operationally, SEA and LEA programs are 
funded* which allows students accepted into the programs to 
enroll in classes free of charge or at minimal cost. University 
programs are courses of study developed by and offered at 
universities for which tuition is charged and no financial 
assistance is systematically provided. In one case the 
distinction is blurred: the North Dakota State University 
program is partially funded by the state. However, because it 
'was developed and is controlled by the university, it is more 
properly considered a university program. 

Not all the programs, which were identified by our sampling 
method, are included in this analysis. Some have been omitted 
because they were unfunded proposals to conduct teacher retrain- 
ing. Others are excluded because they were more concerned with 
training mathematicians and scientists from industry' to become 
teachers, were more properly considered preservice training pro- 
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grams for initial certification, or ceased operation. 4 Table 7 
lists the 11 programs used in this analysis. To included in 
the analysis, a program had to be in operation by September 
1983. 

We mailed program managers sections of an earlier draft of 
this chapter that cited their programs, asking them to verify 
the accuracy of all references. All 11 program managers 
responded and we made any necessary changes. 

SEA programs 

State-level programs to retrain teachers for certification 
in mathematics or science teaching are a recent development. 
State funds are provided either directly to retraining in- 
stitutes, generally schools of education, to furnish retraining 
services to successful applicants certified in other fields, or 
are given directly to teachers in the form of a grant or loan. 
Some states use a combination or both approaches. North 
Carolina, for example, is funding 20 teacher institutes during 
t^€"isiiinm"eTS of 1983 and 1984 to facilitate certification of 
current 7th- and 8th-grade mathematics and scienGe teachers 
who are teaching without appropriate certificates. The 
institutes are offere^d at various teacher training institutions 
throughout the state. Programs were developed under SEA 
guidance by the institutions in conjunction with LEA's and were 
designed to meet local needs. A total of 492 teachers were 
selected from 900 applicants by a committee of institution and 
LEA staff on the basis of principals' recommendations, extent 
and recency of mathematics/science training, and current 
teaching load. (Selection criteria are discussed more in detail 
below.) Participants will receive six college credits 
applicable toward certification for each summer and $35 per day 
for expenses. The state also pays the instructors. (North 
Carolina also provides $1^000 scholarships to mathematics/ 
science teachers to upgrade their skills. Only appropriately 
certified secondary school teachers are eligible for this 
assistance. ) 

Virginia has recently initiated a combined summer insti- 
tute/direct loan program for mathematics teachers. Four univer- 
sities each received $10,000 grants to operate summer institutes 
to retrain certified teachers from other disciplines (or to up- 
grade the training of currently uncertified mathematics teach- 1 
ers, a subject beyond the scope of this chapter). During the 
1983-84 school year, the SEA will also pay $450 of tuition 
costs (in the form of a loan to be ^f orgiven after 2 years 
service) for teachers working toward mathematics certification. 
Virginia is also the only SEA identified as providing a 
retraining grant to an LEA. Fairfax County has received 
grants totalling $15,500 to develop an LEA program to 
conduct a local mathematics institute for its teachers in 
need of certification. 
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Other states rely exclusively either on grants to teacher 
training institutions or on direct loans to teachers. In July* 
1983, Indiana established the "Indiana Teacher Shortage 
Financial Assistance Fund" to provide reeducation grants or 
interest-free loans of up to ?1,000 to certified teachers from 
other disciplines to fulfill certification requirements in 
designated shortage areas (including mathematics and science). 
Kentucky has established two loan programs for mathematics and 
science teachers: a summer program for active* certified 
teachers from other disciplines and a full-time* academic year 
program aimed mainly at undergraduate education students but 
open to certified teachers seeking mathematics or science 
certification. Although loans are made directly to teachers in 
Kentucky* the SEA monitors the program by overseeing the course 
of studies at institutions approved for participation* 
allocating "slots" at participant institutions* and specifying 
criteria to be used by institutions in selecting students and 
recommending them for loan approval. Loans , for full-time 
programs are ?2*500* for summer programs ?833. Academic year 
loans are repayable by a year of teaching mathematics or science 
in a Kentucky public school; summer school loans are repaid by 
one semester's teaching. Florida's program* on the other hand, 
awarded a total of $286*530 to eight state universities to 
provide programs adapted to local requirements either to upgrade 
the skills of certified or uncertified mathematics or 'science 
teachers or to facilitate certification of competent mathemati- 
cians and scientists who do not meet pedagogical requirements 
for certification in the state. ^ 

LEA programs 

Teacher retraining programs have also been initiated at the 
local level. One of the most developed LEA programs is conduct- 
ed by the Houston* Texas* Independent School District. Entitled 
Project Search, it is one component of a broader program to 
remedy a shortage of mathematics and science teachers in Houston 
schools. Project Search recruited 100 teachers from different 
subject areas or teaching levels for retraining in mathematics 
and science. In conjunction with local universities, Houston 
developed and implemented individualized programs to prepare 
elementary teachers for certification at the middle/ junior high 
level and middle/junior high teachers at the secondary school 
level. Course instructors are mathematics teachers, science 
teachers, or supervisors, and classes are taught at Houston 
Community College, a part of the Houston Independent School 
District. Tuition and books are paid for by the LEA, which also 
provides a ?250 -per-course stipend to teachers if they receive a 
grade of B or better. Program participants must commit 
themselves to teach in Houston for 3 years. 

In summer 1982, Fairfax County, Virginia* began a retraining 
program in mathematics for teachers in its district. Through an 
arrangement with a local community college* participants take 
the same series of subject-matter courses as undergraduate 
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majors in mathematics education. Classes are offered at a 
convenient time for teachers who take one course per quarter and 
two during th'e summer. After 2 full years, graduates will have 
fulfilled all requirements for Virginia certification as 
secondary mathematics teachers. Fairfax County has also 
included a master teacher component in tlie program: experienced 
mathematics^ teachers are available for assistance and 
demonstrate model lessons in their classroom. The program is 
offered free of charge, but participants must sign a contract to 
accept any available matherttetics teaching slot after graduation. 

University programs 

Some programs to retrain teachers certified in other fields 
as mathematics or science teachers have been developed at 
universities without funding. Of the programs investigated, all 
were designed to meet state certification standards, and most , 
were for mathematics teachers. Variations exist in the lengthy 
of programs, partially because of the varying state c^tifica- * 
tion requirements but also due to the varying levels of preg ram 
intensity. In the summer of 1983, North Dakota State University 
in Fargo launched a two-summer program in mathematics consisting 
of a series of intensive 2-week courses. At the end of the 
program, teachers will be fully certified to teach mathematics 
at the secondary level in North Dakota, which requires only 16 
semester hours of preparation. Southwest Missouri State Univer- 
sity in Springfield, Missouri/ offera a iaathematics program of 
similar length but more traditional in individual course 
length. Over two summers/ participants take five courses. 
While these are not sufficient for permanent certification in 
themselves (Missouri requires 30 credit hours for certification 
in secondary mathematics), they qualify a teacher for temporary 
certification, which only requires 12 hours. 

Westfield State College in Westfield, Massachusetts, con- 
ducts an intensive retraining program for certification in 
mathematics and mathematics/science. Thirty-nine credit hours 
can be earnod over 9 months of evening and weekend classes. By 
contrast, Temple University in Philadelphia offers a 3-year 
program, which consists of nine courses conferring 27 credits. 

Candidate selection 

*■ Criteria for admitting candidates are expressed most 
formally in the Kentucky program, where a regression equation 
was written by the SEA for use by universities in rank ordering 
candidates. The Kentucky criteria, however, were applied in 
some fashion by most SEA and LEA programs: 

— prior mathematics/science coursework ; most programs 
understandably seek candidates who have already 
accumulated some academic credits in the field; the 
certification payoff is quicker. Kentucky applies 
inverse weighting to the number of hours required for 
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certification; weights range from 1.0 for 10 hours or less 
less to 0.6 for more than 40 hours. 

— undergraduate academic performance ! as dat^rjpined fry 
review of personnel folders or college ^transcripts. 
Preference is given to applicants with more successful 
academic history. 

— past job performance * usually determined by recommenda- 
tions of principals or other supervisors or* as in 
Houston, by a quantitative score on an annual &ssess w 
ment scale. Teachers with higher performance ratings 
are preferred. 

Kentucky expressly includes differential weights for the parti- 
cular field studied (from 1.0 for physics to 0.6 for biology)* 
reflecting the state's need for teachers in different disci- 
plines. It may safely be assumed that other jurisdictions at 
least implicitly consider the perceivod needs of the SEA or LEA. 
in fle eting candidates. Fairfax County administe rs a test used 

by the community college as a screening test for a caXcirtcrs 

course and discourages applicants who score poorly. Indiana 
gives explicit preference to teachers laid off by reason of 
reductions in force. 

The selection process in the university programs investi- 
gated was less stringent. The Temple and the Southwest Missouri 
State programs require previous college mathematics courses for 
admission. ^ Temple University administers an aptitude test to 
applicants. Temple uses 70 percent as a "guideline" for 
admission 'but has no firm cutoff score. Westfield State College 
offers a test to help applicants decide abPut entering the 
program. Essentially participants appear to be self-selected 
with some counseling assistance provided by the universities. 
An official of one university program felt that actual 
performance in the program provided a significant selection 
criterion and viewed the first, course in the program sequence as 
a means of screening out less promising participants. 

Length of program 

We found great variation in the time needed to complete 
retraining programs. Some, like North Dakota State University 
and Southwest Missouri State University, are two-summer 
programs. The Westfield State program takes 9 months, but the 
Temple University program takes 3 years to complete. Several 
factors combine to effect this variation* 

1. certification requirements . A great variation exists 
among states in the amount of preparation required for 
teacher certification. In the states where our training 
programs are located, the number of college credits > 
required for teaching mathematics or science at the 
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2* certification status at program completion * Some 
programs in effect guarantee certification at their 
conclusion* These are typically longer programs or ones 
with high admissions standards* North Carolina aimed at 
getting the teacher sufficiently close to certification 
that the teacher can complete the requirements* 

3* entry standards * Some programs, particularly those 
developed independently by universities, accept 
candidates with fewer previous mathematics or science 
courses than do typical. LEA or SEA programs* 

4. intensity of program * Some programs are quite fast- 
paced and demanding. Westfield State, for example, 
pacJcs 39 credit hours into 9 months of evening, weeJcend, 
and summer classes but attracted few students* Temple 
University spreaos 27 credits over 3 years* 



The actual time needed to complete a program is determined 
by an interaction of these factors, which are largely under the 
the control of the program designer* If a crash program were 
needed arid adequate funds were provided to the institutions to 
develop and conduct the training and to teachers for tuition, 
books, and living expenses, it is liJcely that any of the pro- 
grams described here could be conducted on a full-time, inten- 
sive schedule* If a mbre gradual approach seemed preferable, 
the programs could be adjusted to allow part-time participation 
at a less intense pace* 



HOW EFFECTIVE ARE THE NEW RETRAINING PROGRAMS? 



The ultimate goal of all teacher retraining programs is to 
increase the supply of competent, knowledgeable, and effective 
teachers in the shortage areas of mathematics and science* 
Different criteria of program success could be used* Criteria 
include retention in the program, success in obtaining appro- 
priate teaching credentials (certification), subject-matter 
mastery, continuation in teaching, and quality of teaching 
performance by program graduates* The programs we investigated 
are all too small and too new to provide adequate data on which 
to base solid generalizations* Some anecdotal evidence, 
however, was available* 

We review the available evidence and then discuss both the 
training and retraining findings including the logic of our 
comparisons of the two very different data sources. 

Retention in the program 

Data regarding the rate of program completion were not 
available from all programs, since many are only in their 
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initial stages. In general, programs funded by SEA's or LEA's 
experienced relatively high retention rates in comparison to 
university programs* Houston, for example, has experienced a 
90-percent retention rate* Nearly two-thirds of the first 
cobort are still enrolled in the Fairfax County program* In the 
SEA programs, Kentucky reports retention of 85 out of 90 who 
enrolled in 1982 and Virginia indicates that 62 of 65 teachers 
are continuing in its program* North Carolina reports a high 
percentage of teachers completing the summer training session in 
1983* 7 

By contrast, Temple University expects at least 35 to 
complete the 3-year program but only 46 of the original 83 
enrollees remained after the first year* At Westfield State, 
publicity surrounding the start of the program attracted 400 
inquiries, but the program started with only 15 students, and 
only 5 or 6 are expected to coirtplete the program* The retention 
rate reported from university programs averaged less than 65 
percent and the final figure may be lower since the figures 
include students still in training* 

The higher retention rate experienced by SEA and LEA 
programs is most likely attributable to more careful screening 
of applicants and to the fact that they are offered at little or 
no cost to participants* To some extent also, the fact of being 
selected by their superiors may provide a certain status in the 
eyes of the participants and their peers, which enhanced other 
motives* Participants in university programs lack such 
reinforcement and need a much higher level of motivation to 
continue expending their own time and money* While these 
university programs may seem to have a natural appeal for 
teachers who are either unemployed or in danger of being laid 
off, it is doubtful that teachers who have freshly realized the 
insecurity of their employment future would be- strongly 
mnf ivafftfi t-.n nnH^rfaVft an expensive and demanding program to be 
able to continue in teaching if alternative employment were 
available* Furthermore, higher attrition rates would seem of 
less concern to universities. than to SEA f s and LEA's who must 
view their programs as investments of scarce resources* LEA f s 
and SEA's are naturally more highly motivated to minimize costs 
by screening out applicants unlikely to complete the programs in 
€i reasonable time* The implications of this for federal aid are 
discussed in chapter 6* 

Certi f ication 

The operational goal of all programs is to achieve cer- 
tification for their participants* All programs either qualify 
students for appropriate certificates or, like North Carolina, 
provide sufficient progress toward certification that the student 
may reasonably be expected to complete requirements after 
completion of the program* It is reasonable to assume that most 
program graduates will apply for and receive certification and 




thus swell the numbers of certified mathematics and science 
teachers . 



Houston's Project Search appears to have been remarkably 
successful in alleviating the shortage of certified teachers. 
In 1978, there were 13 secondary science vacancies and 34 mathe- 
matics vacancies. On September 1, 1982, there were only 2 
iflathematics vacancies and 1 science vacancy in Houston. This , 
result cannot be attributed exclusively to Houston's retraining 
prbgram, although 34 of the program's graduates have already 
been placed in mathematics and science classrooms. Houston's 
Second Mile program, of which Project Search is one component, 
is a vigorous, stipend-oriented response to teacher shortages 
and subpar performance. Certified teachers of mathematics and 
science (and certain other fields) receive a $2,000 salary 
differential. Additional stipends are available for attending 
in-service programs or taking subject-related college courses. 
After the introduction of Second Mile , improvements in teacher 
turnover rato and absenteeism, as well as a dramatic reduction 
of vacancies in critical shortage areas, occurred. It would m 
thus be impossible to disentangle the main effects of each 
treatment on teacher shortages, but there would seem little 
doubt that Project Search contributed substantially to 
increasing the supply of qualified mathematics and science 
teachers. It must be noted, however, that such radical changes 
in the system of teacher remuneration as are represented by 
Second Mile may be impossible to implement in LEA's with 
established collective bargaining agreements. 

Subject-matter mastery 

No objective data are available to compare program 
graduates on subject-matter mastery with teachers trained more 
traditionally or with sonie established standard. The director 
of the Temple University program suggested that its graduates 
may not possess mathematical skills equivalent to the graduate 
of a mathematics education program but would be fully qualified 
to teach junior high and basic high school courses. It could be 
argued, however, that graduates of retraining programs would be 
more knowledgeable about recent developments in the field than 
veteran teachers who have not updated their skills for several 
years. 

Quality of teachi ng a nd continu ation in teaching 

The critical criteria by which to judge the success of re- 
training programs are whether they produce qualified, effective 
classroom teachers and a whether these teachers remain in teaching 
for enough years after completing retraining to justify the ex- 
pense of retraining. The retraining strategy assumes that ex- 
perienced teachers from other disciplines are better risks by 
both criteria inasmuch as their prior service is some indica- 
tion of their competence in the classroom and their interest in 
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teaching. (LEA and SEA programs, which use performance evalua- 
tions as factors in the selection process, reduce these risks 
further.) Unfortunately, no objective data are yet available 
from our programs on these ultimate criteria of program 
success- 

Assessing training and retraining programs 

Here we review the effectiveness findings of this and the 
previous chapter and lay out the logic of our comparisons of the 
two very different seta of data. 

We note in chapter 3 that research to date has failed to 
show a straightforward relationship between teachers* knowledge 
and the subsequent learning by their students in mathematics and 
science. Since that research was conducted, increases in the 
supply of uncertified mathematics and .science teachers have been 
reported (see chapter 2), raising the ^possibility that 
uncertified teachers now in or coming into classrooms may 
benefit from training or retraining programs in ways that 
fully certified teachers d o not. This fol lows since the 
inconclusive findings for upgrading training~~may be" based upon 
a group of better qualified teachers. Also, if uncertified 
teachers really have grave weaknesses in subject matter knowl~ 
edge, they have the most to gain from content instruction* 

Whatever the prior preparation and knowledge levels of 
teachers entering retraining programs nay be, we found in this 
chapter that retraining programs succeed in attracting and en- 
rolling applicants, retaining enrollees, and starting to gradu^ 
ate teachers vho are eligible for certification. Retrained 
teachers may or may not, be of the same quality as existing 
mathematics and science teachers. Retraining programs have 
simply been too new to make such comparisons, although that^re- 
search should now be possible, we have no particular reason to 
assume that one brand of certified teacher will be less knowl* 
edgeable or less effective than another, especially since stu* 
dent achievement is not very sensitive to differences in teacher 
knowledge. However, data to test this proposition are not 
available. 

The retraining strategy is certainly promising in terms of 
helping to solve teacher shortage or quantity problems. The 
quantity problem may be more successfully dealt with because it 
is a simpler problem than teacher quality. Certification is a 
process that defines minimal or threshold quality. The shortage 
problem is by definition solved once there are certified 
teachers in the classrooms* Retruining programs seem to be 
helping to increase the supply of certified teachers. " 

As a final note, observe that the line of argument in this 
subsection would not have been possible if programs to upgrade 
teachers had shown evidence of effectiveness in improving 
student achievement. In that case, the burden of proof would 
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have been upon the new retraining programs to demonstrate that 
they could achieve results as good as the (hypothetically ) good 
results of upgrading programs* 

SUMMARY AND CONCLUSIONS 

In this chapter, we reported the results of examining 11 
programs to retrain teachers certified in other fields to teach 
mathematics and science* We discussed the means we used to 
identify retraining programs* We categorized the programs as 
SEA, LEA, or university programs and discussed their different 
approaches to funding, candidate selection, and program dura- 
tion. We suggested criteria for evaluating the effectiveness of 
retraining programs and applied them to the programs in our 
sample* 

The programs we reviewed were limited both in number and in 
years of experience* Nevertheless, some common threads emerge 
from our analysis to suggest the following preliminary findings: 

1* Teacher retraining programs can be expected to increase 

— tfte— gttp ply ^ f ^ ortififf^ " m th gn i^ti p m fl flcif* nge 

teachers* The immediate goal of all the programs we 
reviewed is to provide their participants with most, if 
not all, of the requirements for certification in their 
state as mathematics or science teachers* Early results 
from 11 programs show that teachers apply for admission, 
enroll, and are starting to complete retraining 
programs* There is* little reason to doubt that most 
program graduates will become certified* 

2* University retraining programs tend to suffer from 

lower retention rates and therefore seem an unreliable 
source of newly qualified teachers* The decision to 
enroll and to stay in these programs is probably highly 
sensitive to economic conditions, particularly to the 
threat of teacher layoffs in other fields and the lack 
of alternative employment* In contrast, SEA and LEA 
retraining programs experience higher retention rates, 
in part due to their more careful screening of 
applicants* 

3* Combining funded retraining programs (such as the cur- 
rent ones offered by SEA's and LEA* s) with other 
incentives appears to provid^^an effective strategy* 
Stipends for participating in retraining programs and 
salary increases for teaching ir^ shortage areas may 
offer the quickest solution in communities with severe 
shortages* However, the ppropriate size and mix of in- 
centives can only be det rmined by a careful assessment 
of the extent and urgency of the shortage* Finally, the 
viability of this strategy will be affected by the local 
collective bargaining situation* 
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4. Insufficient information exists at present to predict 
the quality of on-the-job performance by retrained 
teachers. Some have suggested that they will be less 
qualified than traditionally trained teachers, but it 
can be argued that they may be superior in some 
respects . 

5. There was wide variation in the length of time needed to 
complete retraining in the sampled programs: they 
ranged from 9 months to 3 years. Despite this 
variability, program length can be controlled by policy 
intervention based on the urgency of the^teacher short* 
age. In the absence of substantial scholarship and sub- 
sistence payments, short and intensive programs seem to 
attract few students. Thus it is likely that shorter 
programs will prove more costly. 

NOTES * 

*James w* Guthrie and Ami Zusman, "Teacher Supply and 
Demand in Mathematics and Science,*' Phi Delta Kappan , 
September ,1982, pp. 28-*33. See also K. Forbis Jordan, 
Teachers for Precollege Mathematics and Science 
Programs (Washington, D.C. ; Congressional Research Service, 
July 28, 1982). 

^s. Sudman, Applied Sampling (New York: Academic Press, 
1976), pp. 210-11. 

3 Education week , May 18, 1983. 

^The programs not included are the South Carolina SEA, 
Philadelphia, University of Vermont, Fordham Univer- 
sity, Catholic University, and the Greenville (South 
Carolina) LEA. 

^Florida officials informed us that the programs will now 
be handled on a local level instead of the state university 
system. 

6 Southwest Missouri States '/program is designed tor teachers 
already certified to teach elementary schCDl mathematics in 
Missouri- Teachers would have taken previous mathematics 
courses to gain this certification. 

'North Carolina reports th^t 479 out of 488 teachers completed 
v i "ununer training programs. 
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CHAPTER S 
WHAT WILL BE NEEDED TO 



IMPROVE EVALUATIONS OF THE 

QUALITY AND QUANTITY OF MATHEMATICS 

AND SCIENCE TEACHING? 

In this chapter, we note the weaknesses in available data 
about programs to improve mathematics and science teaching as 
well as in assessments of the severity of quality and quantity 
problems in mathematics and science teaching* We then discuss 
some priorities for evaluation under both teacher quality and 
teacher quantity in the context of pending legislation* 

W EAKNESSES IN PAST EVALUATIONS AND 
NEW EVALUATION PRIORITIES 

we have noted throughout this report critical limitations 
in the availability of essential evaluative information* In the 
case of retraining programs, fthe problem lies in part in the 
lack of prior experience with this mechanism* However, we also 
noted no systematic evaluation efforts in this area, perhaps due 
to the current absence of any national level funding in this 
area* in the case of upgrading existing science and mathematics 
teachers, part of the evaluative problem lies in the weakness of 
past evaluation efforts* We believe such efforts should be 
conscientiously improved with any new programs enacted in 
mathematics and science education* 

It was not one of the objectives of this report to assess 
the evaluation function within NSF* Nevertheless, many of the 
available studies we reviewed in chapter 3 were funded by NSF* 
We found that NSF studied the participants in institutes, but 
there was only a modest amount of work initiated in the outcomes 
of NSF institute programs and no comparison of the relative ef- 
fectiveness of different institute approaches". A broader evalu- 
ation program could have helped guide the changes in policies 
and funding decisions that took place over the years at NSF* 

We often found weaknesses in the policy orientation of 
funded evaluations* For example, the single study of non- 
applicants, successful applicants, and rejectees did not analyze 
the data by type of institute* Given the expectation of dif- 
ferent audiences for the different types of institutes, this 
method of analysis may well have obscured important differences 
between institute types* Part of the reason for the very low 
quality of most of the research we did ^locate on the 
effectiveness of NSF institutes was that it was largely unfunded 
and conducted as doctoral dissert? M on research by graduate 
students with all the attendant co * ises of time, sample 
size, and design* 1 
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The proposed legislation to provide federal aid for mathe- 
matics and science education (see chapter 2) does not provide 
for evaluation of all proposed programs. Neither House of 
Representatives 1310 nor Senate 1285 appears to include funding 
for NSF evaluation. 2 Department of Education program evaluation 
could be supported under a provision in the Senate bill author- 
izing the National Institute of Education up to $3 million each 
year for the following research and evaluation activities: 

"(a) a policy analysis of alternative methods to 

improve instruction in mathematics and science; 

(g) an annual evaluation of the programs assisted 
under this title r and 

<C) research on improving teacher trailing, re- 
training, inservice training, and retention, as 
well as the development of curriculum and 
materials in the fields of mathematics and 
science . "3 

The limited evaluation support provided in these bills com- 
bined with budgetary pressure limiting the availability of other 
agency funds lead us to conclude that it may never be known 
whether or not programs enacted are effective. This may be of 
concern to the Congress especially given the major information 
gaps we discuss in earlier chapters. These gaps include the 
lack of documentation of whether or not there is a mathematics 
and science teacher shortage and direct evidence on the quality 
of mathematics and science teachers. 

Early in any newly enacted program* there will be a need to 
inform the Congress of basic evaluative information regarding 
che programs funded. NSF and the Department of Education could 
plan early studies on the allocation of funds (including 
substate allocations) , needs identified! activities undertaken, 
an assessment of what teachers receive what type of training, 
and the match between available funds and training needs. There 
are irany different service delivery mechanisms for training 
authorized under the bills, which makes a coordinated national 
level effort to assess resource needs especially important. We 
saw evidence that by 1964 the NSF institutes were addressing 
popular but perhaps marginal needs such as institutes for very 
recent graduates cf teacher preparation programs and institutes 
aimed at providing teachers with master's degrees. 

Evaluation_p riorit3.es for teacher^uality 

The first priority for evaluation in respect to teacher 
quality issues in our view is a needs assessment of teacher 
quality. 4 The data available, as we pointed but in chapter 2, 
are achievement test scores of teachers in general but not of 
science and mathematics teachers. We know little about the 
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knowledge levels of mathematics and science teachers and whether 
or not they are out-of-date in their subject matter area* How * 
much can we learn from the National Teacher Examinations and 
similar state tests, which have recently been required for 
entering teachers in many states, especially in the South? New 
nationwide data collection on teacher quality may be necessary. 
If the results are to give a national portrayal of the status * 
of mathematics and science teaching quality, collection of 
comparable data across the country would be a real necessity. 

We know that many new mathematics and science teachers lack 
certification upon entry, but we know little about how many 
courses they do have in science and mathematics, how many stay 
teaching science and mathematics for more than one or two years, 
and what subsequent coursework these teachers take in science 
and mathematics. In short, we known very little concretely 
about how serious a problem we hav** in science and mathematics 
teacher quality and to what extent it is a nationwide problem. 

If the proposed legislation is enacted, there will be more 
dollars and a greater variety of remedies than in past pro- 
grams. This will provide more opportunities for alternative 
strategies, but it also increases the need to plan nationwide 
.evaluations that address the effectiveness of the remedies. 

Figure 1 (see page 60) serves to conceptualize the factors 
that could be considered in program ©valuations that are intended 
to improve teacher quality and subsequent student achievement in 
mathematics and science. We begin this discussion by noting that 
the level of student achievement in mathematics and science — the 
outcome entry at the far right of figure 1 — is better documented 
than the other topics we have reviewed. The testing program of 
the National Assessment of Educational Progress funded by the 
Department of Education has provided a useful yardstick of 
student achievement in science and mathematics at ages 9, 13, and 
17. Data from standardized^ tests — especially in mathematics but 
also in science — provide other indicators of how much students 
know. 5 The National Science Board Commission on Precollege 
Education in Mathematics, Science and Technology has recommended 
the development of a national assessment that allows national, 
state, and local comparisons in contrast to the national level 
reporting under the National Assessment of Educational 
Progress. This effort would presumably replace the National 
Assessment * 

The measurement *of the other factors or influences shown in 
figure 1 is much less developed. We know too little about how 
effective teaching can increase student achievement or how a 
factor such as effective teaching interacts with other factors 
such as student characteristics, school organization, and school 
culture* Another complication is that multiple programs are 
funded to increase teaching effectiveness and they are intro- 
duced at many points of influence. Programs to increase the 
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quality of teaching materials in the schools as welT~~as in- 
service training programs will be occurring simultaneously with- 
in schools. Evaluation needs to be spread over all programs if 
impacts are to be traced and if any assessment is to be made of 
relative effects and consequent needs for priority-setting. 

Evaluation priorities for teacher quantity 

The first priority in our view regarding the shortage of 
mathematics and science teachers is to address the problem of 
seriously flawed available data on the extent of the shortage. 
This is not a matter of purists quibbling over minor weaknesses in 
data, rvs we noted in chapter 2, the shortage statistics are based 
on opinion surveys, partial examinations of the total prob-lem 
(such as the number of newly certified graduates in each field 
each year), and imprecise data on numbers of teaching vacancies. 
Efforts to assess the impact of new federal legislation would seem 
to require good data on the extent of shortages and changes over 
time. We are encouraged that the National Academy of Sciences is 
active in examining this 

problem, but we know of no planned recurring nationwide data 
collection that will meet the needs we have citedi above. 

■ 

Figure 2 serves to conceptualize the factors* which affect 
the number or quantity of mathematics and science teachers. 
Student aid programs funded by federal* state* or university 
sources affect both initial decisions of students to prepare to 
be teachers (the first column of the figure) and subsequent- 
decisions on entering the field of teaching (middle column of 
the figure). ' The latter decisions are presumably affected by 
the nature of the payback requirements contained in student aid 
programs. A second possible evaluation area — in addition to the 
measurement of the shortage as just discussed — concerns the impact 
of scholarships. It may be useful to expand upon the work done 
in this area for previous loan forgiveness and service payback 
programs by the Congressional Research Service.^ 

A third possible area of evaluation — also represented i n 
the middle column of figure 2 — is an exomination of successful 
and unsuccessful efforts in filling mathematics and science 
teacher vacancies. Some observers have suggested that it is the 
dual effects of low pay and low morale and status of teachers 
that inhibit recruitment and retention., Even if only one of 
these barriers can be reduced, some maintain, the effects on 
teacher supply will be substantial. 

A fourth important consideration is an examination of why 
science and mathematics teachers stay in teaching. The right- 
hand column of figure 2 shows some of the factors that can 
influence teachers * decisions to remain in the classrooms. It 
is important to learn what makes some schools more successful 
than others in retaining teachers. Houf effective are new 
federal efforts aimed at teacher recognition in this regard? 
What it is that attracts former teachers to reenter teaching 
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after ohildrearing and other careers is a potentially important 
influence on the quantity of mathematics and science teachers. 
This element is represented with the dashed lines at the bottom 
of figure' 2. 

Finally, in chapter 2 we cited a variety of programmatic 
attempts to increase the quantity of mathematics and science 
teachers. If the evaluation of teacher retraining programs is 
at all representative of the state of evaluation of all programs 
for increasing the quantity of teachers, the evaluation gaps are 
serious, we need to know more about who is attracted to such 
programs, conditions of programs that enhance their appeal, the 
comprehensiveness of retraining courses compared with standard 
coursework, and the effectiveness and limitations of retrained 
teachers* 



The information gaps in mathematics and science education 
are extensive. We have little confidence in available data on 
the extent and nature cf teacher shortages in mathematics and 
science. Data on the quality of mathematics and science 
teachers — and whether or net the quality of technical teaching 
has declined — are similarly flawed. We also lack a body of 
solid evaluations oieafederal programs to improve mathematics and 
science education. We argue that the limited evaluation support 
in proposed mathematics and 'science education legislation may 
mean that it will not be known whether or not programs enacted 
under this legislation are effective. In the next chapter, we 
summarize our observations. 



^Marilyn N. Suydam and Alan Osborne, The Status of Pre-College 
Science, Mathematics, and Social Science Educations 1955-1975 , 
vol . 2 , Mathematics Education ( Columbus , Ohio : Center for 
Science anp Mathematics Education, Ohio State University, 1977), 
p. 170. In his review of 138 documents on institute effects in 
1974, Helgeson found that 63 or 46 percent were theses or disser- 
tations. See Stanley I/. Helgeson, "Impact of the National 
Science Foundation Teacher Institute Program,' 1 University of 
Minnesota, Minneapolis, 1974, p. 2. 

2 The House bill authorizes $5 million each year for NSF research, 
but this does not appear to include authorization to evaluate 
NSF programs. H.R. 1310, March 2, 1983, sec. 105 and 107(b). 

3 S. 1285, May 16, 1983, sec. 212(b)(2). 

4Some similar suggestions are presented in Nancy B. Borkow and 
K. Forbis Jordan, The Teacher Workforce: Analysis of Issues and 
Options for Federal Action (Washington, D.C. : Congressional 
Research Service, November 7, 1983), p. 28. 
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^Lyle V* Jones, "Achievement Test Scores in Mathematics and 
Science," Science , July 24, 1981, pp* 412-16* 

6 National Science Board Commission on Precollege Education in 
Mathematics, Science and Technology, Educating Americans for the 
21st Century (Washington, D*C*: National Science Board, National 
Science Foundation, 1983), pp* 11-12* 

?Jim Stedman, The Experience with Loan Forgiveness and Service 
Payback in Federal and state Student Aid Programs (Washington, 
D*C. : Congressional Research Service, 198377 
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CHAPTER 6 



OBSERVATIONS 

In this chapter, we consolidate our major observations on 
improving the quality and quantity of mathematics and science 
teaching, 

QUALITY AND QUANTITY PRIORITIES 

Our analysis raises questions about approaches to upgrade 
the quality of mathematics and science teaching that have 
substantial teacher training compon4*at?* This report suggests 
that programs geared at upgrading existing mathematics and 
science teachers may not produce results in terms of improved 
student achievement. 

Upgrading training may be best focused upon uncertified 
teachers now in or coming into the mathematics and science 
' classr^Sms, Past research may not be germane to this group 
since the appearance of a substantial number of uncertified 
teachers may be a recent phenomenon. In addition, the 
uncertified teachers represent the group in greatest need of 
subject matter training. This approach would differ from the 
prior NSF strategy and would require careful planning for 
successful NSF implementation. 

Solutions to the teacher quality and quantity problems are 
to some degree in opposition. Calls for increasing quality 
standards exclude more teachers from the classroom and delay the 
entry of others. 

Programs to deal wijjth teacher quantity or shortage problems 
offer more pro/nise than efforts to improve teacher quality, we 
found that taacher retraining programs can increase the supply 
of certified teachers. Retention data from a few programs 
suggest that it will be possible to graduate a reasonable number 
of those enter. 

The quantity or shortage problem may be more successfully 
dealt with because it is a simpler problem. Historically, we 
have defined qualification for teaching by certification. In 
this sense, our quality and quantity categories become a bit 
blurred. Certification is a stage that defines minimal or 
threshold quality. The shortage problem is by definition solved 
once there are certified teachers in the classrooms. 

Teacher quality is a separate and seemingly more intract- 
able issue. Once teachers have passed some threshold of know- 
ledge through certification, can we further increase their sub- 
ject matter knowledge? And if we do, does it help students? To 
treat completion of institutes or other upgrading training as 
the equivalent of completion of certification would be to ignore 
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the absence of much evidence of a demonstrable educational value 
of such training in improving student achievement, 

From an evaluation perspective, the point is that the 
teacher quantity and quality questions are different in level of 
difficulty, While criteria of completion of training are ade- 
quate for the former, the latter requires more rigorous crite- 
ria, The implication of this observation is that the certifi- 
cation or quantity issue is more immediately susceptible to 
intervention than improving teacher quality, 

While our findings on teacher upgrading do not encourage a 
substantial investment in such activities, there could be other 
teacher quality approaches that are more promising, We did not 
examine the approaches oriented toward equipment, materials, and 
program improvement that are part of the remedies the Congress 
is considering, We also note that school quality improvement 
stategies such as magnet schools and gifted and talented student 
programs are alternatives, 

HOW MUCH TIME DOES IT TAKE 
TO RETRAIN TEACHERS? 

There was considerable interest in the Congress over the 
issue of how much time it takes to retrain teachers for 
mathematics and science classrooms, In practice, we found wide 
variation in the length of time to complete retraining programs 
ranging from 9 months to 3 years, The North Carolina SEA program 
requires only two summers because it has the objective of getting 
teachers close to certification, leaving the final steps to the' 
teacher after the program, Shorter programs also insist on more 
completed mathematics or science courses for admission, The SEA 
and LEA programs in particular are likely to require more prior 
coursework in order to lower their costs and maximize student 
retention in the programs, In the absence of substantial 
scholarship and subsistence payments, full-time programs seem to 
attract few students, Program length is thus highly variable 
but can be controlled by policy intervention, It is likely that 
shorter retraining programs will be more costly through both 
scholarship and Subsistence costs, 

STRUCTURING EFFECTIVE RETRAINING PROGRAMS 

University retraining programs tend to suffer from lower 
retention rates and therefore seem an unreliable source of newly 
qualified teachers, The decision to enroll and to stay in these 
programs is probably highly sensitive to economic conditions, 
particularly to the threat of teachers layoffs in other fields 
and the lack of alternative employment, 

Combining LEA or SEA funded retraining programs with other 
incentives appears to provide an effective retraining strategy, 
Stipends for participating in retraining programs and salary 
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increases for teaching in shortage areas may offer the quickest 
solution in communities with severe shortages. The appropriate 
size and mix of incentives can only be determined by an assess- 
ment of the extent and urgency of the shortage, and the viabil- 
ity of this strategy will be affected by the local collective 
bargaining situation . 

The SEA and LEA retraining programs experience higher 
retention rates than university programs, in part due to their 
more careful screening of applicants. The SEA's and LEA's are 
concerned about attrition since thoy are paying for the retrain- 
ing. This suggests that the strategy in the proposed mathemat- 
icj and science legislation of requiring linkage between univer- 
sities and school districts in training programs may be 
productive * 

PROCESS-PRODUCT APPROACHES 

The efforts of process-product researchers to identify 
effective teaching behaviors and to develop teacher training 
programs around those findings offer promising possibilities. 
The approach has been limited to mathematics and reading and 
could be attempted in science teaching. Also, the bulk of the 
effort has been conducted at the elementary grade levels. It 
may be useful to devise a secondary level research program 
beginning with the identification of effective secondary level 
teachers and correlates of successful teaching. 

G APS IN EVALUATIVE INFORMATION 

The gaps in evaluative information are so serious that we 
conclude that even the question of whether or not there is a net 
nationwide shortage of mathematics and physical science teachers 
is inadequately documented today. Similarly, data on the quality 
of existing mathematics and science teachers are based on 
surrogate measures of quality such as certification status or on 
test scores of teachers in all fields rather than mathematics and 
science teachers. 

There are two corresponding informational needs, which re- 
sult. First, with respect to the size of any shortage, is the 
need for adequate data &t both the national and state levels on 
the extent of the shortfall by subject each year. Second, with 
respect to quality, is the need for an adequate assessment of the 
knowledge levels of mathematics science teachers and whether 

or not they are out-of-date in thair subject areas. These 
information gaps both hinder the enactment and administration of 
effective federal remedies and retard appropriate evaluation of 
the success of new federal remedies . 

We found little evaluation of the past NSF institute 
programs and essentially no national level evaluation of the 
various retraining initiatives. The limited evaluation support 
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provided for programs in pending bills to aid mathematics and 
science education coi: ^ned with budgetary pressure limiting the 
availability of other agency funds lead us to conclude that it 
may never be known whether or not programs are effective* 
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SOME METHODOLOGICAL ASPECTS OF NSF 

INSTITUTE EVALUATION _. 

Selection of an effectiveness criterion * 

In our examination of NSF institute effectiveness, the 
criterion we selected was impact on student academic achieve- 
ment* Student achievement has been ihe major educational 
outcome variable among education evaluations. It represents the 
most fundamental goal of pur education system and, directly or 
indirectly, of most education programs at the precollege level. 
There is a long history of the development and application of 
measures of student achievement in order to measure student and 
school progress. Thu*;, achievement can be measured with greater 
validity and reliability than some other outcome measures, 
Finally, a focus on student achievement is consistent with the 
concerns raised by numerous commissions during 1983 calling for 
improvements in American^ schooling. The chief practical 
disadvantage is the time and expense in locating, sampling, and 
testing students of the former JLnstitute participants. The gap 
in time between the acceptance of applicants into institutes and 
the subsequent achievement of students requires data analysis 
that carefully accounts for other extraneous factors that will 
affact student achievement or an experimental design that 
randomizes the influence of other factors* 

Other outcomes of institute effectiveness have been 
studied* Some investigators have examined student attitudes 
toward mathematics or science* The difficulty lies in the 
questionable validity of measures of student attitudes*! 

The impact of NSF institute attendance upon numerous 
teacher outcomes has also h % studied* Measures of teachers* 
attitudes toward mathematics and science or toward teaching have 
been studied. In our opinion, they share the technical problems 
of the comparable student measures* Since the rationale for 
such criteria lies in desired impacts on student behavior 
{higher student achievement, interest in science, coursetaking ) , 
it seems more direct to use student measures* A second teacher 
outcome is subject matter competence or knowledge of mathematics 
or science attributable to institute attendance* This can be 
measured through a variety of standardized or constructed 
tests* While short-term changes are relatively simple to 
measure (pre- and postcomparisons during the period of the 
institute), longer-term gains in knowledge seem much more 
significant than information Recalled at the end of a course. 
Studies of longer term effects do not seem to have been 
conducted. 

A third teacher outcome concerns the retention of upgraded 
mathematics and science teachers in the classrooms or in 
education^related occupations. Retention rates can be measured 
rather directly through questionnaires mailed to former 
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institute participants and comparison teachers. However, the 
salary and morale problems discussed in chapter 2 may be 
changing patterns of teaching careers and teacher retention. 
One could argue that if teachers leave the field of teaching 
after about five years, this is not necessarily undesirable 
since short-term teaching careers may be a realistic goal for 
recruiting future mathematics and science teachers. 

Fourth, some have argued that institutes served to increase 
teachers' professional stature in thair schools. Fifth, insti- 
tutes may lead to increased professional activity afterwards. 
While these outcomes are desirable, they are not likely to be 
considered significant enough to justify an institute program. 

Next we describe our procedures for searching for studies 
of the academic effects of NSF institutes. 

Search methods and selection criteria 
for studies 

The procedure we followed in searching for studies of the 
impact of NSF institute training on the subsequent achievement 
of the participants' mathematics or science students included 
the following: 

1. the ERIC data base. 

2. bibliographies from a 1974 synthesis and two syntheses done 
for NSF in 1977. 2 

3. interviews with major authors of earlier works, researchers 
and evaluators in the field, as well as officials from NSF 
and the Department of Education* 

4. recent issues of the Journal of Research in Science Teaching, 
Science Education , and School Sc ience and Mathematics for 
articles that may not yet have been entered in the ERIC system. 

The candidate studies identified through this procedure were 
obtained throug!. a lengthy interlibrary loan and ERIC document 
ordering process except for some that were not avail* ole. 

Studies of student achievement were selected for inclusion 
if they met the following criteria: 

1. grades K-12. 

2. conducted within the United States. 

3. published 1950 or later. 

4. exclude dissertations. 3 

5. NSF must have funded all or part of the institute. 

6. An experimental group must include a teacher institute or 
some form of in-service science or mathematics-related 
training. {"In-service" includes any training aftei 
initial certification.) 

7. Control or comparison group must be included. 

8. Student academic achievement must be an outcome variable. 
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9. If the institute involved study of a new mathematics or 

science curriculum (as many institutes did, particularly in 
the later years of the program) , there must have been some 
design or analytic procedure to attempt to separate effects 
of training from effects of introducing the new curriculum 
into the participating teachers' classes. 

While this is an extensive list of criteria, it is an ap- 
propriate one, given the information we need on the effective- 
ness of teacher upgrading. We found two studies showing gains 
in student achievement, but with no means of comparison there 
was no reason either to consider tMe gains anything beyond 
normal expe^ked learning or to attribute the c;ains to institute 
participation by the students 1 teachers. We found four studies 
that approached our criteria. Two were funded by other agencies 
(National Institute of Education and the Environmental Protection 
Agency). The third involved unspecified training of four teach" 
ers in a new mathematics curriculum (SMSG), and the fourth con- 
cerned a program operated by a university and several school 
districts involving the training of local resource leaders vho 
subsequently designed a variety of local training programs. 
These four studies represent an unusual assortment. We are 
uncertain how similar they were to activities funded by NSF. 
Applying our criteria, we found only the one study reported 
by Victor Willson and Antoine Garibaldi (see appendix II ). 

NOTES 

ISee Marilyn N. Suydam and Alan Osborne, The Status of Pre- 
College Science, Mathematics, and Social Scie n ce Education; 
1955-1975 , vol . 2, Mathematics Education (Columbus , Ohio; 
Center for Science and Mathematics Education, 1977), ch. 3; 
Stanley L. Helgeson, "Impact of the National Science Founda- 
tion Teacher Institute Program," University of Minnesota, 
Minneapolis, 1974; Hugh Munby, "Thirty Studies Involving the 
'Scientific Attitude Inventory 1 i What Confidence Can We Have in 
This Instrument?" Journal of Research in Science Teaching , 20;2 
(1983), 141-62. 

2 Helgeson; Suydam and Osborne; Stanley. Helgeson et al., The 
Status of Pre-Colleqe Science, Mathematics and Science Education: 
1955-19 75, vol. 1, Sc ience Education (Columbus, Ohio; Center for 
Science and Mathematics Education, f977), ch. 3. 

3The use of dissertations was not practical given staffing and 
time limits. The senior author of a major series of meta- 
analyses in science education concluded that dissertations tend 
to have smaller average effect sizes than journal articles. 
However, the differences are small .enough that "all indications 
are that a complete or random sampling of studies is ot crit- 
ical*" Ronald D. Anderson, "A Consolidation and Apj,. ^isal of 
Science Meta-Analyses, 11 Journal of Research in Science Teaching , 
20:5 (1983). 499 and 50 7 . 
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NOTES ON THE STUDY DESIGN , 

METHODOLOGY , AND RESULTS OF THE 

WILLSON AND GARIBALDI RESEARCH 

The Willson and Garibaldi research examined the achievement 
of students whose teachers had high participation in NSF insti- 
tutes (3" 14 institutes attended), medium participation (1-2 
institutes) , and low participation (no institutes) . 1 Principals 
of sampled schools randomly selected one science or mathematics 
teacher who in turn was asked to complete a questionnaire, take 
the applicable National Teacher Examination, and select one 
class at random for testing. Science students took the Test of 
Achievement in Science (consisting of 40 items selected from the 
National Assessment of Educational Progress) while mathematics 
students took the Mathematics Achievement Test (composed of 40 
items from the National Longitudinal Study of Mathematical 
Abilities). Two forms were developed for each subject area, one 
for 8th and one for Hth graders. A total of 346 teachers and 
classes participated in science and 211 in mathematics. Science 
classes were from schools sampled in Wyoming, South Dakota, and 
Mississippi regions while mathematics classes were in California 
and Indiana regions. ^ Analysis of covariance results show NSF 
participation was a significant factor in high school science 
and mathematics achievement. Region was also a significant 
factor in science achievement* A covariate — teacher achievement 
test score — was included because the teachers attending 
NSF institutes were believed to be more academically able, as we 
discussed in chapter 3- In the present context, including 
teacher achievement as a covariate could have removed effects of 
additional learning gained at the institutes. The covariate was 
nonsignificant in all cases, suggesting that the teachers' 
knowledge of science or mathematics was not related to their 
students' achievement. NSF participation was not significant 
for 8th-grade student achievement in either science or 
Mathematics • 

This study does suggest positive effects of NSF institutes 
the high school level for both science and mathematics 
achievement. The results do not seem attributable to higher 
ability of the classes taught by NSF institute participants. 
Contingency tables of amount of NSF institute pai ticipation by 
teacher ratings of the ability of the sampled classes were 
prepared for each of the four groups (two grade levels and two 
subjects). There were no significant associations, indicating 
that teachtfrs with more NSF institute experience were not 
systematically assigned to more able classes. 

Although this study meets our screening criteria presented 
earlier in this section, it still hac weaknesses due to the 
nonexperimental nature of the design used. Self-selection is 
one major threat to the validity of the study. It is possible 
that a professionalism or drive for self-improvement led 
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teachers to apply to attend NSF inst itutes and it was that 
initiative rather than any effect of the institutes that had 
some effect on student achievement. In a similar fashion, it is 
possible that schools with mere teachers applying for NSF 
institutes are more achievement oriented , more demanding in 
their recruitment of tochers * and in other ways produce higher 
achieving students * 

We noted that the authors found no association between 
institute attendance and teachers rating of the ability gr :>up of 
the class selected for testing. A study design involving pre- 
and post-tests would be a much stronger control over possible 
differences in initial ability of the students of institute 
attenders compared* with nonattenders. The authors note that if 
NSF had followed an experimental stategy, a stronger analysis 
than this post hoc approach would have been possible. The 
approach actually used is also open to mortality problems. If 
teachers receiving training had more subsequent opportunities* 
which over a period of years led the better teachers to leave 
the classrooms or to migrate to better paying jobs in more urban 
aroas * a subsequent ext v nation of teacher effects could be said 
to underestimate train 1 effects. It is certainly true that 
this sample was rural l€ t s than 20 percent of schools were in 
citios of 50*000 population or more. Finally, we note that 
there was no on-site data quality control* Sampling of 
teachers* classes, and students was all done by school staff 
with only instructions from the researchers. The teacher tests' 
were self-administered and school personnel gave the student 
tests* This is not to say that this study was poorly done. It 
was the best we found but still has significant limitations. 

NOTES 

^Victor L. Willson and Antoine M. Garibaldi* "The Association 
Between Teacher Participation in NSF Institutes and Student 
Achievement*" Journal of Research in Science Teaching* 13:5 
(1976)* 431-39* and "The Effect of Teacher Participation in NSF 
Institutes Upon Student Achievement*" revised* University of 
Minnesota* Minneapolis* September 30, 1974. 

^Regions were not contiguous with states. For example* Denver 
was part of the Wyoming region. 
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AGENCY COMMENTS AND OUR RESPONSE 

All four agencies responded to our request for comment on a 
draft of this report: NSF, the Department of Education, the 
Office of Science and Technology Policy, and the Office of 
Management and Budget. We first address the major comments that 
appeared in some form in two or more agency replies. We then 
deal with the additional comments. The letters are reprinted 
at the end of this appendix. 

MAJOR COMMENTS 

The acjencies generally characterized the report as useful 
and addressed to issues of major importance. There are two main 
areas of comment. 

First . All agencies expressed some * concern about our 
finding little evidence that institute types of programs for 
science and mathematics teachers have produced improved student 
achievement. The Department of Education believes that such 
programs "can improve teaching effectiveness" and anticipates "a 
substantial requirement for new subject-matter mastery for most 
mathematics and science teacherr." The Office of Management and 
Budget and NSF question the different criteria used to measure 
the effectiveness of training for upgrading teachers and . 
retraining programs. NSF is concerned that our report may be 
counterproductive to the efforts of other national reports 
calling "for enhancing "the attractiveness of the teaching 
profession and for increasing the educational standards for_ 
teachers, especially their subject matter knowledge." The Office 
of Science and Technology Policy feels that we have a "clear bias 
against upgrading the quality of existing mathematics and science 
teachers." 

. 

Our response . We are not arguing that efforts to upgrade 
existing mathematics and science teachers should stop. We assume 
that individual science and mathematics teachers will take subject 
matter courses and that many will be assisted financially by their 
LEA's and other sources. Similarly, we wish to advance rather 
than oppose efforts to improve science and mathematics education 
in the public schools. But many remedies have been suggested for 
improving mathematics and science education. One major task 
facing education policymakers is to identify the remedies in which 
they will invest time and resources, the remedies in which they 
w.'ll make little or no investment and the remedies in which they 
will invest o*i a pilot basis. Our analysis suggests that programs 
geared to upgrading existing science and mathematics teachers have 
not been shown to produce results in terms of improved student 
achievement* This is not a "bias" but the results of our 
evaluation. 

Only two factual arguments are raised against our position. 
1. NSF argues that we should have included the results of a 
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meta-analysis of studies of science teacher characteristics as 
related to student outcomes. Only two of the studies 'included in 
that meta-analysis were of academic institutes* however. They 
were in the group of studies we considered for selection t see 
appendix I ) . Also* we did not report on studies linking the 
amount of teacher preparation in science to student outcomes, 
which is the major finding of that research cited by NSF. 
Rather, we looked at studies linking teacher knowledge — whether 
acquired through ability, coursework, self- teaching — and student 
academic achievement. In any event, the correlations found in 
the study NSF cites were very small? the correlations of academic 
institute training with student achievement were less than half 
of the correlations for variables such as "masculinity . " As the 
author notes, "The most striking overall characteristic of the 
results of this meta-analysis is the pattern of low correlations 
across the large number of variables involved. "1 The NSF argu- 
ment is inconsistent in the sense that NSF notes "the limitations 
of the correlational studies, " which we cite. But the study NSF^ 
wanted us to report is a meta-analysis* which is also a correla- 
tional method of analysis, msf admits that the effect sizes in 
the research they cite are small but feels that is "what is to be 
expected" in such settings. Thus* in the final analysis, NSF 
agrees with us that these teacher characteristics have not been 
shown have a moderate or reliable effect upon student 
outcomes • 

2. The argument is raised by NSF and thfe Office of 
Management and Budget that we use different criteria ( relying upo 
student achievement gains in assessing teacher upgrading but upon 
the completion of certification by teachers in assessing 
retraining programs. Since upgrading programs are for addressing 
teacher qualify problems, while retraining programs are for the 
purpose of solving the shortage or quantity problems* we believe 
that different criteria are both appropriate and necessary. The 
shortage of mathematics and science teachers is defined as the 
absence of teachers certified in these fields. Our research 
suggests that retraining programs offer early evidence of success 
in remedying this shortage. The comparative quality of teachers 
graduating from retraining programs compared with traditional 
study is an interesting question but one that extends beyond the 
question of remedying teacher shortages and into teacher quality 
concerns. We discuss these issues in some detail in chapters 4 
and 6. 

NSF and the Office of Management and Budget both seem to be 
suggesting that teacher upgrading may help increase teacher 
retention or make the profession more attractive to others*, We 
respond that efforts su2h as the NSF program for Presidential 
? *ards for Teaching Excellence in Science and Mathematics as well 
as the Honors Workshop for Teachers of Science and Mathematics 
seer,* to be more dire'ctly lin?ted to objectives of enhancing the 
atcractiven^si of the teaching profession. (We have not studied 
the actual effects of these programs but ^efer here to the 
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intended direct effects of programs.) Also see our discussion of 
the pending science and mathematics legislation and other 
remedies such as increased pay -for teachers {chapter 2). 

In conclusion, the few facts' presented by the agencies do 
not give us reason to alter our findings. 

Second . Three agencies agree on the need for improved 
evaluation and statistics (see chapters 5 and 6). The Office of 
Management and Budget says the federal government "should take 
steps to ensure effective ongoing evaluation" in mathematics and 
science education. The agency agrees that there is now a "lack 
of information available to shape fully effective Federal 
programs for improving mathematics and science education." NSF 
concurs that there is a "need for reliable and regularly 
collected data on the supply and demand of science and mathe- 
matics teachers and we also recognize the need for the National 
Science Foundation to improve its program evaluation effort. 1 ' 
The Department of Education agrees that a systematic study that 
examines all of the relevant factors in teacher shortages is 
necessary. The Department of Education favors data-gathering 
when "timely and economical, and when suitable methodology 
exists" but believes that no "suitable methodology presently 
exists that can be applied to Figure 1." 

Our response . We are pleased at the level of agreement and 
hope that the agencies will begin establishing priorities for 
evaluation and other data collection. Figure 1 is intended to 
conceptualize the factors that may influence teacher quality and 
subsequent student achievement. Its purpose is to organize 
the relevant variables and help to identify, data gaps and future 
evaluation priorities. We are not proposing a comprehensive 
study of all factors included in figure 1. 

ADDITIONAL COMMENTS 

The Department of Education 

First. The Department of Education believes that there is 
evidence of current mathematics and science teacher shortages 
sufficient to justify federal action. The department character- 
izes our report as containing "no contradictory evidence* nor does 
it make an effort to structure a definition of what constitutes a 
shortage that merits federal action." 

We do not take a position^ on whether or not federal action 
in mathematics and science education is justified. What we do 
argue — *> "*d what the department fails to rebut — is that data ade- 
quate to determine whether or not there is a new nationwide 
shortage of mathematics and physical science teachers do not 
exist . 
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Second. The Department of Education also holds 

"a different view from that expressed in the report on the 
contrary effects on teacher shortages of increased gradua- 
t ion requirements and declining enrol lmentSr and believe 
teacher shortages may increase." 

We argue that increased graduation requirements would seem to 
lead to increased Leacher shortages although various factors may 
negate such additional shortages (chapter 2). We note that one 
contrary factor could be continued declines in hi£h school 
enrollments* as projected by the- Department of Education. 

National Science Foundation 

First. NSF objects to our treatment of "process-product" 
research characterizing that research as resting on "tenuous 
evidence" and as "irrelevant to the immediate problem of high 
school science and mathematics." 

We ourselves note these "limitations of process-product 
research in the text but cite the woik because applying the 
process^product methodology to exploring effective teaching in 
secondary school mathematics and science classes could be a 
valuable contribution. We also note other areas of recent 
research ^ See chapters 3 and 6. 

Second. NSF also states that our report is incomplete in 
that it ignores "practices in some projects and programs that 
represented significant departures from early 'institute 1 
practices. " 

We are not clear what material NSF would like added. This 
report is not intended to be a comprehensive evaluation of all NSF 
teacher education activities. 

Office of Science and Technology Policy 

The agency finds with regard to our finding on upgrading 
teachers that what "starts as a qualified statement in chapter 3 
becomes stronger in succeeding chapters and ends up as an 
assertion in the digest." 

The digest conclusion — "GAO does not find evidence that 
training programs to upgrade existing mathematics and science 
teachers will produce results in terms of improved stm *nt 
achievement" — deals with the lack of evidence on program 
effectiveness. The central conclusion in chapters 3 and 6 
considers the implications of this lack of evidence, namely that 
"programs- geared at upgrading existing mathematics and science 
teachers may not produce results in terms of improved student 
achievement. " 
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Office of Management and Budget 

The agency observes that the need for additional science and 
mathematics teachers varies from district to district and that the 
most appropriate methods for overcoming shortages differ across 
districts* 

We made similar arguments in chapters 2 and 4, 

NOTES 

^-Cynthia Ann Druva and Ronald D, Anderson, "Science Teacher 
Characteristics by Teacher Behavior and by student Outcome: A 
Meta-Analysis of Research*" Journal of Research in Science 
Teaching, 20:5 (1983), p. 478, 
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UNITED STATES DEPARTMENT OH EDUCATION 

WASH J NO TON IK &2it2 



assjsh an r stt rltak v 

J-OR bDl>CATtONAL KtStARt H AND IMPROVhMhN 1 



Mr, Richard FogeX 
Director 

Human Resources Division 

United states General Amounting Office 

Washington* D,C, 20548 

Dear Mr, Fbgel: 

Cn behalf of the Secretary of Education, I wish to thank you for the 
opportunity to review and comment upon the proposed GAD report, N Wew 
Directions for Federal Programs to Aid Mathematics and science Teaching," 
It has relevance for various Departmental activities and initiatives, 
and it contains much useful information. We find, however, that we are 
unable to concur in two of the report's major findings, and must qualify 
our concurrence in a third. Our summary response is organized along 
these findings, 

(1) We believe well designed programs to upgrade existing secondary 
school mathematics and science teachers can improve teaching effec- 
tiveness. Such programs should emphasize a suitable mix of subject- 
matter mastery, and pedagogical-skills and classreonwnanagement mastery 
as may be required by the particular characteristics of the program's 
teacher purticirvints. Given present changes in the nature of the 
economy ai i new conceptualizations and discoveries in the fields of 
mathematics and science, we anticipate a substantial requirement for new 
subject-matter mastery for most mathematics and science teachers. This 
view is broadly held in the education and science education coimunities, 
and is included in the reoorrmenda tior -s of both the National Commission 
on Excellence in Education and the Marional Science Board's Commission 
on Preco liege Education in Mathematics, Science and Technology. CXir 
reading of the proposed GfO report ir, that it contains some evidence in 
support of this view and little or none to contradict it. If declining 
test scores are a current policy concern/ we find upgrading as we define 
it here to be an appropriate national policy, 

(2) We believe sufficient evidence of current teacher shortages exists 
to justify federal action, Among this evidence are the strong efforts 
presently being made by States and localities that involve substantial 
expenditures in retraining programs to correct the deficit* The pro- 
posed GRO report contains no contradictory evidence, nor does it make an 



79 




APPENDIX III 



APPENDIX 



- 2 - 



effort to struc r uro a definition of what constitute* a shortage that 
merits federal action. The Administration's proposal to assist the 
States in correcting the current deficit is contained in the Science and 
Mathematics Teacher Develo]*nunt Act of 1983 (II, R, 1324 and S, 706). tfe 
also bold a different view from tliat expressed in the report on the 
contrary effects on teacher shortages of increased graduation require- 
ments and declining enrollments and believe teacher shortages may In- 
crease, A systematic study that examines all of theVelevant factors is 
necessary to assist in better defining the precise nature, extent and 
especially the need for long term action, 

(3) While we beliove more data, better data and wove and better program 
evaluations are generally desirable* we observe that they are not with- 
out cost, nor — in the historical record ^- without limit in their 
utility to policy-makers, tfe do favor data-gathering, including statis- 
tical data-gathering, and program evaluations when these are timely and 
economical, and when suitable methodology exists. For example, we are 
unaware that suitable methodology Presently exists that can be applied 
* to Figure 1, page 5-6, of the report in a way rhat can be expected to 
provido unambiguous guidance for policy makers. 

If any clarification of those conroents is required, I may be reached at 
254-8251, 




Sincerely, 



Donald J, Senese 



* 



*6A0 note: page 60* 
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NATIONAL SCIENCE FOUNDATION 
WASHINGTON DC 20550 




January 13, 1984 



OFFICE OF THE 
DIRECTOR 



Mr. J. Dexter Peach- 

Director, Resources, Community, and 

Economic Development Division 
United States General Accounting Office 
Washington, D.C. 20548 

Dear Mr. Peach: 

Tha>. A you for your letter of December 15, 1983, enclosing an advance draft 
of the proposed report "New Directions for Federal Programs to Aid 
Mathematics afid Science Teaching." He find the issues addressed to be of 
major importance and appreciate having the opportunity to comment. 

n 

He f4nd that we are in substantial agreement with some of the observations 
and recommendations made in the report. Specifically, we agree with the 
position stressing the need for reliable and regularly collected d^ta on 
the supply and demand of science and mathematics teachers and we also 
recognize the need for the National Science Foundation to improve its 
program evaluation effort. 

Yet, we find other positions — indeed, the main positions — unfortunate 
and actually counterproductive to the national effort to improve science 
and mathematics education. Several national reports call for enhancing the 
attractiveness of the teaching profession and for increasing the*educa- 
tional standards for teachers, especially their subject matter knowledge, 
as essential to improving the effectiveness of teacfjirvg and the participa- 
tion and achievement of students in scienpe and mathematics. The position 
taken in the report undermines the momentum that has: been building at the 
Federal, State and local levels. ] 

I have enclosed additional comments made by members of the Foundation's 
staff. In view of the points raised, I believe the present draft of the 
report is not only confusing, but may lead in the wrong direction with 
regard to public policy formulation. 



Si ncerely* 




Edward A. Knapp 
- Director 



Enclosure 
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NSF Staff Comments on the GAO Report, 
"New Directions for Federal Programs 
to Aid Mathematics and Science Teaching" 



The report's position that "programs geared at upgrading existing 
mathematics and science teachers are not likely to produce results 1n terms 
of improved student achievement* * *" — even as qualified within the text 
runs counter to the national consensus* and the limited data presented 
within the report do not warrant such a sweeping conclusion* 

* Furthermore, in section 3* the conclusions state that general research 
"failed to show any consistent relationship between the extent of teachers' 
knowledge and subsequent student learning* * * [and] the studies of second- 
ary school science and math learning in the I960' s and 1970* s failed to 
show any straightforward relationship between teachers' knowledge and 
subsequent student learning* 11 These conclusions may be unwarranted and 
based on an incomplete consideration of recently published research on this 
matter* For example, an extensive analysis of 65 studies of science 
teacher characteristics as related to teacher behavior and student outcomes 
{see the Journal of Research in Science Teaching. Volume 20 L1983D » 
pp. 467-479) concludes: 

o "Student outcomes are positively associated with the preparation of 
the teacher, especially science training * * *; 

o "* * * an intellectual orientation on the part of teachers is 
positively related to cognitive student outcomes* * *; [and] 

o "The relationship between teachers' training in science and 

cognitive student outcome is progressively higher in higher level 
science courses*" 

These authors use the powerful quantitative techniaue of meta-analysis to 
compare effect size across these 65 studies (75% of which were performed 
between 1966 and 1975)* They point out that the effect sizes are 
relatively small but what is to be expected in such complex multivariate 
settings* Consequently, the conclusions as stated in the draft report are 
mi sleading* 

Given the limitations of the correlational studies cited in the GAO report 
and considering the complexity of the multivariate system being considered* 
we believe the SAO position on teachers' knowledge to be unwarranted and to 
jeopardize the overall policy goals (Federal, State and local) for 
improving teacher subject matter competence as a means to improving the 
education of students* 



*GAO note: chapter 3* 
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It also seems to us that holding out "process-product" research results 
with elementary school teachers as offering*the means of improving student 
* achievement (pp* 3-23 to 3-26) not only rests on tenuous evidence but 1s 
irrelevant to the immediate problem of high school science and mathematics, 
which is of central concern in the report* As a result, its inclusion may 
be misleading* 

Other aspects of the report are also troubling to us, and we list a few 
without going into detail* 

o The report uses criteria Inconsistently* For example, teacher 
upgrading programs are discussed in terms of student achievement, 
but the retraining programs for teachers in other fields to become 
mathematics and science teachers are judged by the number of 
participants immediately or eventually certified* 

o The report is Incomplete on NSF teacher education programs 1n 
ignoring practices in some projects and programs that represented 
significant departures from early "Institute" practices, 



*GAO note: pages 34-36. 
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EXECUTIVE OFFICE OF THE PRESIDENT 
OFFICE OF SCIENCE AND TECHNOLOGY POUCY 



WASHINGTON, OC 3000 



January 18 , 1984 



Dear Mr* Peach: ^ 

This is in response to your recent letter to Dr* Keyworth 
transmitting the draft report *New Directions for Federal 
Programs to Aid Mathematics and Science Teaching" for our 
review and comment. 

As a general comment, the report is longer than necessary to 
make its points. Only the most dedicated readers are likely 
to sift through all the detail in order to verify the statements 
made in the digest. 

A major flaw of the report is its clear bias against upgrading 
the quality of existing mathematics and science teachers* What 
starts as a qualified statement in Chapter 3 becomes stronger 
in succeeding chapters and ends up as an assertioh in the digest* 
The evidence to support this assertion is' no stronger than the 
counter evidence which is noted but suppressed. A superficial 
reading of the report is not likely to uncover this point* 

Given these difficulties with logic and objectivity, the report 
in its present form will be misleading if used for policy 
formulation* 



Mr* J. Dexter Peach 
Director 

Resources, Community, and 

Economic Development Division 
U*S* General Accounting Office 
Washington, D.C* 20548 




Sincerely, 
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EXECUTIVE OFFICE OF THE PRESIDENT 
OFFICE OF MANAGEMENT AND BUDGET 

WASHINGTON, D.C. 20503 



JAN 2 0 1984 



Mr. William J. Anderson 
Director, General Government 

Division 
United States General 

Accounting Office 
Washington, D.C. 20548 

Dear Mr . Anderson: 

Thank you for giving us an opportunity to comment on your 
draft report, "New Directions for Federal Programs to Aid 
Mathematics and Science Teaching.** 

The report provides very useful background on the National 
Science Foundation* s previous in-service teacher training 
programs and on a select sample of teacher retraining 
programs now in operation across the country. While we agree 
with many of the report* s observations about the advantages 
and disadvantages of in-service training and retraining, we 
remain reluctant to draw a conclusion that either of these 
approaches is super ior . 

Your report assesses the relative effectiveness of these two 
strategies using different standards. It uses student 
achievement gains to measure the effectiveness of in-service 
training programs like those administered by NSF in prior 
years. The report uses an unrelated measure, the increase in 
certified math and science teachers, to evaluate the teacher 
retraining, approach. Of equal interest would be; (1) how 
in-service training programs influence teacher supply (either 
by increasing teacher retention or making the profession more 
attractive to others); and {2) the effectiveness of retrained 
teachers in raising student achievement in science and math. 

The findings in Chapter 5 of your report support our general 
belief that there is a lack of information available to shape 
fully effective Federal programs for improving mathematics 
and science education. The resource levels proposed by this 
Administration for science and mathematics education programs 
reflect the judgment that the Federal Government should 
proceed cautiously in making investments in pre-co^lege 
mathematics and science education. It should also take steps 
to ensure effective ongoing evaluation. 
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The observations in your report are quite consistent with 
this Administration's view that: 

o the primary responsibility for generating resources 
and designing programs to meet local education needs 
should remain at the State and school district 
level* 

o the Federal government can be most effective as a 
catalyst in this area and should encourage more 
direct action by State and local governments and 
businesses* 

As your report makes clear, the need for additional science 
and mathematics teachers varies considerably from district to 
district and is apparently non-existent for some. Similarly, 
the most appropriate methods for overcoming teacher shortages 
differ across districts; the Houston example, while 
effective, is clearly not appropriate for many districts 
where collective bargaining agreements would interfere. 

We hope you* will consider these comments in developing your 
final report* Thank -you again for the opportunity to 
respond. 



Sincere 




ignt, Jr. 
iuty Director 



(973185) 
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